Fraud detection on streaming transaction graphs (STGs) faces challenges on the catastrophic forgetting of previously learned fraud patterns when adapting to evolving patterns. Although some Graph Continual Learning (GCL) approaches mitigate this issue by storing and revisiting historical samples, practical storage constraints prevent them from fully preserving previous patterns. In this work, we propose STG-DGR, a streaming GNN model with diffusion-based generative replay that generates synthetic samples to retain previously learned patterns without storing real samples. The generation of replay samples for STGs faces two key challenges: (1) Heterogeneity challenge of generating STG samples with discrete adjacency table, user features, transaction features, and transaction timestamps. (2) Dependency challenge of capturing bottom-up dependencies across layers in STG samples. To address these challenges, STG-DGR integrates two novel components: (1) a Computational Subgraph Processor (CSP) that transforms heterogeneous STG samples into well-organized hierarchical subgraphs, and (2) a Diffusion-based Subgraph Generator (DSG) that captures the bottom-up dependencies using a novel Transformer-based Hierarchical Denoising Network (THDN), and generates synthetic replay samples that preserve these dependencies. Extensive experiments on four streaming fraud detection datasets demonstrate STG-DGR's superiority in reducing forgetting and improving accuracy over nineteen state-of-the-art baselines.

Federated recommender systems enable collaborative model training while keeping user interaction data local and sharing only essential model parameters, thereby mitigating privacy risks. However, existing methods overlook a critical issue, i.e., the stable learning of a generalized item embedding throughout the federated recommender system training process. Item embedding plays a central role in facilitating knowledge sharing across clients. Yet, under the cross-device setting, local data distributions exhibit significant heterogeneity and sparsity, exacerbating the difficulty of learning generalized embeddings. These factors make the stable learning of generalized item embeddings both indispensable for effective federated recommendation and inherently difficult to achieve. To fill this gap, we propose a new federated recommendation framework, named Federated Recommendation with Generalized Embedding Learning (FedRecGEL). We reformulate the federated recommendation problem from an item-centered perspective and cast it as a multi-task learning problem, aiming to learn generalized embeddings throughout the training procedure. Based on theoretical analysis, we employ sharpness-aware minimization to address the generalization problem, thereby stabilizing the training process and enhancing recommendation performance. Extensive experiments on four datasets demonstrate the effectiveness of FedRecGEL in significantly improving federated recommendation performance. Our code is available at https://github.com/anonymifish/FedRecGEL.

Recommender systems have become increasingly influential in shaping user behavior and decision-making, highlighting their growing impact in various domains. Meanwhile, the widespread adoption of machine learning models in recommender systems has raised significant concerns regarding user privacy and security. As compliance with privacy regulations becomes more critical, there is a pressing need to address the issue of recommendation unlearning, i.e., eliminating the memory of specific training data from the learned recommendation models. Despite its importance, traditional machine unlearning methods are ill-suited for recommendation unlearning due to the unique challenges posed by collaborative interactions and model parameters. This survey offers a comprehensive review of the latest advancements in recommendation unlearning, exploring the design principles, challenges, and methodologies associated with this emerging field. We provide a unified taxonomy that categorizes different recommendation unlearning approaches, followed by a summary of widely used benchmarks and metrics for evaluation. By reviewing the current state of research, this survey aims to guide the development of more efficient, scalable, and robust recommendation unlearning techniques. Furthermore, we identify open research questions in this field, which could pave the way for future innovations not only in recommendation unlearning but also in a broader range of unlearning tasks across different machine learning applications.

Hash-based collaborative filtering (Hash-CF) approaches recently employ efficient Hamming distance of learned binary representations to accelerate recommendations. Benefiting from its probabilistic nature, Variational Autoencoder (VAE) enables robust Hash-CF with stronger generalization ability. However, VAE-based Hash-CF still faces two challenging problems: 1) Traditional VAE urges the latent variables of different users (or items) to fit a unified and monotonous prior distribution, and lacks considerations for distinctive characteristics of users (or items). The obtained representations of users and items with slight individual differentiation may further weaken the performance of Hash-CF for subsequent personalized recommendations. 2) Hash-CF under the VAE framework requires discrete optimization on latent Bernoulli distributions, which are discrete and NP-hard to optimize. In this paper, we propose a Dual Discrete Collaborative Filtering (DDCF) approach, including a cluster-enhanced representation generation module and a CNF-enabled discrete optimization module. The former module mainly develops cluster-aware latent space to generate discriminative representations for users or items with significantly different characteristics. The latter module employs Continuous Normalizing Flow (CNF) to achieve discrete optimization on latent Bernoulli distributions steadily and effectively. Extensive experiments conducted on multiple real-world datasets demonstrate the superiority of our DDCF compared with the state-of-art methods in terms of effectiveness and efficiency.

Cross-Domain Recommendation (CDR) can effectively alleviate the data sparsity issue in the recommendation system by transferring the source domain knowledge to the target domain. Many CDR methods try to find a mapping of latent embeddings of the overlapping nodes (e.g., the overlapping users) from source domains to target domains. However, due to the randomness of parameter initialization and model training, the embeddings from different domains may differ a lot and the inter-domain mapping can be difficult to learn. To this end, in this work, we propose a novel Similarity-Preserving Cross-Domain Recommendation ( SimCDR ) framework. Instead of trying to learn the inter-domain mapping, SimCDR aims to preserve the intra-domain similarities of users, such that the users that are close in the source domain are still close in the target domain. Specifically, we first verify that the intra-domain similarities of user embeddings can represent domain knowledge well. Then, based on this finding, we propose the SimCDR framework, which adds the auxiliary task of predicting similar users to preserve the intra-domain similarities of the source domain user embeddings, i.e., the source domain knowledge. Besides being distinct in knowledge transfer, SimCDR also supports multiple source domains and embedding compression, making it more suitable for real-world applications. We conduct thorough experiments that verify the effectiveness of the proposed method.

Personalized recommender systems are critical for enhancing user engagement across a range of digital platforms. However, conventional approaches rely heavily on centralized data collection, raising significant privacy concerns. Federated recommender systems (PFRS) address these concerns by decentralizing model training, ensuring user data privacy. Despite the progress, existing methods still struggle with capturing the multi-faceted nature of user and transferring global knowledge effectively. In this work, we propose FedMUR, a novel federated recommendation framework that models user representation as a Gaussian mixture distribution, capturing users' multi-faceted characteristics. Each Gaussian component corresponds to a distinct interest facet, with adaptive mixture weights representing the user's preference intensity toward each facet. To facilitate knowledge transfer, FedMUR constructs global consistent prototypes that encode shared behavioral trends across users via popularity-weighted optimal transport. These prototypes enhance local models by injecting global shared patterns into personalized representation learning. Extensive experiments across several real-world datasets demonstrate that FedMUR significantly outperforms existing state-of-the-art federated recommendation systems.

Session-based Recommendation (SBR) explores dynamic user interests based on short anonymous sessions. The performance of traditional ID-collaborative SBR models is constrained by the limited session length. To this end, we focus on Multi-modal Session-based Recommendation (MMSBR), which aims to leverage modality knowledge to promote short-term user interest modeling. Previous studies cannot solve the MMSBR problem well due to issues as: (I1) Modal-invariant and specific dependencies are difficult to disentangle and unify. (I2) Multiple modal transitions produce mutual effects. (I3) Modal-aware personalized preference discrepancy exists. Thus, we propose a modality-aware diffusion-based framework MDSD with subspace disentanglement, which decouples multi-modal collaborations and provides a new paradigm that integrates modality representation generation into next-item prediction. We first employ modality-disentangled consistency graphs based on affinity semantics to extract the unified modal subspace. Then we design a cross-modal contrastive attention fusion that explores cross-modality interplay, enhancing modal alignment and consistency. To interpret personalized interest discrepancy, we realize the preference-guided multi-modal diffusion, which combines modality-aware interest generation with prediction based on collaborative signals, thus providing comprehensive recommendation results. Extensive experiments on real-world datasets demonstrate the effectiveness of MDSD.

Federated learning (FL) enables collaborative training on decentralized data while preserving privacy by avoiding direct data sharing. However, long-tailed data distributions are common in real-world applications, often resulting in biased models with degraded performance. In FL, this issue is further complicated by privacy-preserving constraints and non-IID data, highlighting the importance of federated long-tailed learning (Fed-LT). To tackle the challenges of Fed-LT, we propose Synthetic Feature-based Decoupled training (SFD) method. To improve local training, we introduce Adaptive Bi-Branch Learning (ABBL) to jointly enhance feature representation and decision boundary learning for non-IID long-tailed data. To mitigate global model bias while preserving privacy, we propose Statistically Aligned Feature Synthesis (SAFS) for global classifier fine-tuning. SAFS constructs privacy-preserving synthetic features that approximate the global feature distribution. These synthetic features enable the global classifier to be fine-tuned without requiring clients to share local training data, thereby alleviating the model bias caused by non-IID long-tailed data. Extensive experiments show that SFD effectively addresses the challenges of Fed-LT and achieves superior performance on Fed-LT datasets.

Industrial recommender systems usually train models incrementally to grasp recent interests of users. However, a fundamental issue of these incremental updated models is their tendency to overfit current data while neglecting past information. Specifically, we have observed that the data distribution of real systems exhibits periodic drifts, leading to periodic fluctuations of prediction bias. To alleviate the above bias fluctuations while minimizing the loss of recent interests, we propose TPIA, a Training-free approach for Periodic Interest Augmentation in incremental recommendation. Specifically, after the latest model is trained, we first calculate the importance score of each model in the previous period. Then, we merge these models based on the importance scores. To minimize information loss due to interference of parameters during model merging, we further develop a method for trimming redundant and abnormal parameters. Offline experiments on both public and private datasets demonstrate the effectiveness of TPIA. It has also been deployed on a large-scale industrial recommender system, and has shown a notable 1.61% increase in CVR and a 1.97% increase in CPM, along with enhanced stability in prediction bias.

Federated Recommendation (FedRec) enables joint training across a large number of clients without centralizing user interaction data. However, existing FedRec methods overlook two key challenges, i.e. (1) sufficiently explore the global item embedding space, and (2) effectively achieve local and global collaboration. The former is caused by client sparsity, which leads to suboptimal item embeddings and subsequently impacts the global item embedding in both the dimension and sample views. The latter arises from the lack of modeling the relative importance of local and global contributions to personalized user preferences. To address the above challenges, we propose FedIAR which contains two modules, i.e., item embedding dual-view exploration and adaptive local-global fusion. The first module enhances the global item embedding by reducing redundancy in the dimension view and capturing latent item relationships in the sample view, improving representational capacity. The second module enables the adaptive fusion of local and global item embeddings based on the user preference representation, achieving personalized optimum for recommendation. Extensive experiments on six datasets demonstrate the effectiveness of FedIAR in improving federated recommendation performance.

Federated graph learning involves training graph neural networks distributively on local graphs and aggregating model parameters in a central server. However, existing methods fail to effectively capture and leverage the inherent global structures, hindering local structural modeling. To address this, we propose Federated Graph Factorization (FedGF), which enhances structural knowledge via privacy-preserving graph factorization. Specifically, FedGF includes three modules, i.e., global structure reconstruction (GSR), local structure exploration (LSE), and global-local structure alignment (GLSA). Firstly, GSR factorizes client graphs into a series of learnable graph atoms and conducts reconstruction to capture the globally shared structure. Then, LSE explores the local structure, mining potential but unrevealed connections within client subgraphs. GLSA further aligns the global and local structure to alternatively refine the graph atoms and GNN model, enhancing the overall structural modeling. Extensive experiments on six datasets consistently validate the effectiveness of \modelname.

Cross-Domain Recommendation (CDR) has been widely investi- gated for solving long-standing data sparsity problem via knowl- edge sharing across domains. In this paper, we focus on the Multi- Modal Cross-Domain Recommendation (MMCDR) problem where different items have multi-modal information while few users are overlapped across domains. MMCDR is particularly challenging in two aspects: fully exploiting diverse multi-modal information within each domain and leveraging useful knowledge transfer across domains. However, previous methods fail to cluster items with similar characteristics while filtering out inherit noises within different modalities, hurdling the model performance. What is worse, conventional CDR models primarily rely on overlapped users for domain adaptation, making them ill-equipped to handle scenarios where the majority of users are non-overlapped. To fill this gap, we propose Joint Similarity Item Exploration and Overlapped User Guidance (SIEOUG) for solving the MMCDR problem. SIEOUG first proposes similarity item exploration module, which not only obtains pair-wise and group-wise item-item graph knowledge, but also reduces irrelevant noise for multi-modal modeling. Then SIEOUG proposes user-item collaborative filtering module to aggregate user/item embeddings with the attention mechanism for collaborative filtering. Finally SIEOUG proposes overlapped user guidance module with optimal user matching for knowledge sharing across domains. Our empirical study on Amazon dataset with several different tasks demonstrates that SIEOUG significantly outperforms the state-of-the-art models under the MMCDR setting.

With the escalating privacy concerns in recommender systems, attribute unlearning has drawn widespread attention as an effective approach against attribute inference attacks. This approach focuses on unlearning users' privacy attributes to reduce the performance of attackers while preserving the overall effectiveness of recommendation. Current research attempts to achieve attribute unlearning through adversarial training and distribution alignment in the statistic setting. However, these methods often struggle in dynamic real-world environments, particularly when considering scenarios where unlearning requests are frequently updated. In this paper, we first identify three main challenges of current methods in dynamic environments, i.e., irreversible operation, low efficiency, and unsatisfied recommendation preservation. To overcome these challenges, we propose a Pluggable Attribute Unlearning framework, PAU. Upon receiving an unlearning request, PAU plugs an additional erasure module into the original model to achieve unlearning. This module can perform a reverse operation if the request is later withdrawn. To enhance the efficiency of unlearning, we introduce rate distortion theory and reduce the attack performance by maximizing the encoded bits required for users' embedding within the same class of the unlearned attribute and minimizing those for different classes, which eliminates the need to calculate the centroid distribution for alignment. We further preserve recommendation performance by constraining the compactness of the user embedding space around a reasonable flood level. Extensive experiments conducted on four real-world datasets and three mainstream recommendation models demonstrate the effectiveness of our proposed framework.

The sparse portfolio optimization (SPO) problem is increasingly crucial in portfolio management, focusing on selecting a few stocks with the potential for strong market performance. However, sparse portfolio strategies often face significant short-term drawdowns during periods of market volatility. To this end, a news-driven portfolio strategy offers valuable insights to capture sudden market changes. Nevertheless, it encounters two main challenges:how to reasonably map the relationships between news and stocksandhow to effectively utilize the irregular timing of news releases. To tackle the SPO problem in fluctuating markets while addressing these challenges, we propose a novel news-driven sparse portfolio strategy, named SPIN. Specifically, SPIN not only leverages industry-specific group structures existing among stocks for a more reasonable news-stock mapping and models news sequential patterns based on our devised novel news-driven forecaster to handle the irregularity of news releases. We rigorously prove that SPIN achieves a sub-linear regret. Extensive experiments on three real-world datasets demonstrate SPIN's superiority over state-of-the-art portfolio strategies in terms of cumulative wealth and short-term drawdowns.

With the growing privacy concerns in recommender systems, recommendation unlearning is getting increasing attention. Existing studies predominantly use training data, i.e., model inputs, as unlearning target. However, attackers can extract private information from the model even if it has not been explicitly encountered during training. We name this unseen information as attribute and treat it as unlearning target. To protect the sensitive attribute of users, Attribute Unlearning (AU) aims to make target attributes indistinguishable. In this article, we focus on a strict but practical setting of AU, namely Post-Training Attribute Unlearning (PoT-AU), where unlearning can only be performed after the training of the recommendation model is completed. To address the PoT-AU problem in recommender systems, we propose a two-component loss function. The first component is distinguishability loss, where we design a distribution-based measurement to make attribute labels indistinguishable from attackers. We further extend this measurement to handle multi-class attribute cases with efficient computational overhead. The second component is regularization loss, where we explore a function-space measurement that effectively maintains recommendation performance compared to parameter-space regularization. We use stochastic gradient descent algorithm to optimize our proposed loss. Extensive experiments on four real-world datasets demonstrate the effectiveness of our proposed methods.

In the dynamic environment of multimedia-sharing platforms like X (formerly known as Twitter) and TikTok, multimedia recommendation systems have been widely used to help users discover items of interest. However, traditional approaches often fall short, when the item modalities are incomplete, a common issue in real-world scenarios. To this end, we introduce the unified heterogeneous Hypergraph construction for the Incomplete multimedia REcommendation ( HIRE ), a novel framework designed to jointly learn a heterogeneous hypergraph and perform accurate recommendations under incomplete scenarios. HIRE first initializes the unified heterogeneous hypergraph for modality completion and employs self-supervised learning aligned with the contrastive text-centered view for multimedia recommendation. Such integration effectively handles the challenges posed by incomplete modalities, leading to improved recommendation accuracy. Furthermore, we find that the hypergraph directly learned from the HIRE is a dense structure which can be inaccurate and coarse. Therefore, we devise the HIRE framework with Sparse constraint named HIRES , which uniquely integrates optimal transport and a \(\ell_{2,1}\) -norm to refine the hypergraph structure. Our extensive experiments across various datasets demonstrate the superiority of HIRES in addressing incomplete modalities, establishing it as a powerful tool for personalized multimedia recommendations.

Federated Recommendation (FedRec) has been widely applied recently for realizing privacy preservation in recommender systems. However, due to direct uploads of model gradients from all clients, FedRec is vulnerable to potential poisoning attacks. In this article, we focus on the targeted model poisoning attacks in FedRec, which aims to raise the exposure ratio of specific target items by generating poisoned gradients to influence global training. Challenges emerge when implementing this kind of attack. On the one hand, simulating authentic users on the malicious clients for downstream poisoning is hard when access to prior knowledge is limited. On the other hand, distinguished item attributes and personalized user preferences require the attack to be adaptive to complex distributions. To this end, we propose a novel attack DuAda with two modules, i.e., dummy user simulator and adaptive distribution attacker . The dummy user simulator is designed to generate malicious users with characteristics similar to real users, which exploits authentic user representations and preference labels simultaneously through two-stage inversion optimization. The attacker first extracts heterogeneous distributions by a special multi-prototype clustering method, and then conducts adaptive attacks from both explicit and implicit promotion perspectives. The explicit promotion raises the prediction scores of target items based on the inherent characteristics, while the implicit promotion imbues them with the features of popular items. Targeted at our proposed attack method, we also design a merged adaptive defense mechanism to fight against DuAda and conduct defensive experiments. Empirical studies on four real-world datasets demonstrate the effectiveness and interpretability of DuAda.

Cross-Domain Recommendation (CDR) have received widespread attention due to their ability to utilize rich information across domains. However, most existing CDR methods assume an ideal static condition that is not practical in industrial recommendation systems (RS). Therefore, simply applying existing CDR methods in the industrial RS environment may lead to low effectiveness and efficiency. To fill this gap, we propose DIIT, an end-to-end Domain-Invariant Information Transfer method for industrial cross-domain recommendation. Specifically, We first simulate the industrial RS environment that maintains respective models in multiple domains, each of them is trained in the incremental mode. Then, for improving the effectiveness, we design two extractors to fully extract domain-invariant information from the latest source domain models at the domain level and the representation level respectively. Finally, for improving the efficiency, we design a migrator to transfer the extracted information to the latest target domain model, which only need the target domain model for inference. Experiments conducted on one production dataset and two public datasets verify the effectiveness and efficiency of DIIT.

The sparse interactions between users and items have aggravated the difficulty of their representations in recommender systems. Existing methods leverage tags to alleviate the sparsity problem but ignore prevalent logical relations among items and tags (e.g., membership, hierarchy, and exclusion), which can be leveraged to enhance the accuracy of modeling user preferences and conducting recommendations. To this end, we propose to extract logical relations among item tags from existing tag taxonomies and exploit the individual strengths of the Poincaré and the Lorentz models in hyperbolic space for logical relation modeling towards enhanced recommendations. Moreover, we find that the logical relations directly extracted from existing tag taxonomies can be inaccurate and coarse. Therefore, we further devise innovative consistency-based and granularity- based weighting mechanisms based on user behavior patterns for data-driven logical relation mining that can be jointly optimized along with recommendations in an end-to-end fashion. Extensive experiments on four real-world benchmark datasets show drastic performance gains brought by our proposed framework, which constantly achieves an average of 8.25% improvement over state-of-the-art competitors regarding both Recall and NDCG metrics. Insightful case studies further demonstrate that our automatically refined logical relations are highly accurate and interpretable.

Estimating individual treatment effects (ITE) from observational data is challenging due to the absence of counterfactuals and the treatment selection bias. Prevalent ITE estimation methods tackle these challenges by aligning the treated and controlled distributions in the representational space. However, two critical issues have long been overlooked: (1)Mini-batch sampling sensitivity (MSS) issue, where representation distribution alignment at a mini-batch level is vulnerable to poor sampling cases, such as data imbalance and outliers; (2)Inconsistent representation learning (IRL) issue, where representation learning within a unified backbone network suffers from inconsistent gradient update directions due to the distribution skew between different treatment groups. To resolve these issues, we propose CE-RCFR, a Robust CounterFactual Regression framework for Consensus-Enabled causal effect estimation, including a relaxed distribution discrepancy regularizer (RDDR) module and a consensus-enabled aggregator (CEA) module. Specifically, for the robust representation alignment perspective, RDDR addresses the MSS issue by minimizing unbalanced optimal transport divergence between different treatment groups with a relaxed marginal constraint. For the accurate representation optimization perspective, CEA addresses the IRL issue by resolving the consistent gradient update directions on shared parameters within the backbone network. Extensive experiments demonstrate that CE-RCFR significantly outperforms the state-of-the-art methods in treatment effect estimations.

The service system involves multiple stakeholders, making it crucial to ensure fairness. In this paper, we take the example of a typical service system, the recommender system, to investigate how to identify and tackle fairness issues within the service system. Recommender systems often exhibit bias towards a small user group, resulting in pronounced unfairness in recommendation performance, specifically the User-Oriented Fairness (UOF) issue. Existing research on UOF faces limitations in addressing two pivotal challenges: CH1: Current methods fall short in addressing the root cause of the UOF issue, stemming from an unfair training process between advantaged and disadvantaged users. CH2: Current methods struggle to unveil compelling correlations among users in sparse datasets. In this paper, we propose a novel Hypergraph Convolutional Network for User-Oriented Fairness, namely HyperUOF, to address the aforementioned challenges. HyperUOF serves as a versatile framework applicable to various backbone recommendation models for achieving UOF. To address CH1, HyperUOF employs an in-processing method that enhances the training process of disadvantaged users during model training. To addressCH2, HyperUOF incorporates a hypergraph-based approach, proven effective in sparse datasets, to explore high-order correlations among users. We conduct extensive experiments on three real-world datasets based on four backbone recommendation models to prove the effectiveness of our proposed HyperUOF.

Federated Recommendation (FedRec) is popularly investigated in personalized recommenders for preserving user privacy. However, due to the distributed training paradigm, FedRec is vulnerable to model poisoning attacks. In this paper, we focus on the targeted model poisoning attack against FedRec, which aims at effectively attacking the FedRec via uploading poisoned gradients to raise the exposure ratio of a multi-target item set. Previous attack methods excel with fewer target items but suffer performance decline as the amount of target items increases, which reveals two perennially neglected issues: (i) The simple promotion of prediction scores without considering intrinsic collaborations between users and items is ineffective in multi-target cases. (ii) Target items are heterogeneous, which requires discriminative attacking users and strategies for different targets. To address the issues, we propose a novel Heterogeneous Multi-target Transfer Attack framework named HMTA which consists of two stages, i.e., (1) diverse user agent generation and (2) optimal multi-target transport attack. The former stage leverages collaboration-aware manifold learning to extract latent associations among users and items, and develops a differentiable contrastive sorting to generate user agents from both difficulty and diversity scale. The latter stage conducts poisoning in a fine-grained and distinguishing way, which first completes distribution mapping from target items to generated user agents and then achieves a hybrid multi-target attack. Extensive experiments on benchmark datasets demonstrate the effectiveness of HMTA.

User cold-start recommendation aims to provide accurate items for the newly joint users and is a hot and challenging problem. Nowadays as people participant in different domains, how to recommend items in the new domain for users in an old domain has become more urgent. In this paper, we focus on the Dual Cold-Start Cross Domain Recommendation (Dual-CSCDR) problem. That is, providing the most relevant items for new users on the source and target domains. The prime task in Dual-CSCDR is to properly model user-item rating interactions and map user expressive embeddings across domains. However, previous approaches cannot solve Dual-CSCDR well, since they separate the collaborative filtering and distribution mapping process, leading to the error superimposition issue. Moreover, most of these methods fail to fully exploit the cross-domain relationship among large number of non-overlapped users, which strongly limits their performance. To fill this gap, we propose User Distribution Mapping model with Collaborative Filtering (UDMCF), a novel end-to-end cold-start cross-domain recommendation framework for the Dual-CSCDR problem. UDMCF includes two main modules, i.e., rating prediction module and distribution alignment module. The former module adopts one-hot ID vectors and multi-hot historical ratings for collaborative filtering via a contrastive loss. The latter module contains overlapped user embedding alignment and general user subgroup distribution alignment. Specifically, we innovatively propose unbalance distribution optimal transport with typical subgroup discovering algorithm to map the whole user distributions. Our empirical study on several datasets demonstrates that UDMCF significantly outperforms the state-of-the-art models under the Dual-CSCDR setting.

Cross-Domain Recommendation has been popularly studied to resolve data sparsity problem via leveraging knowledge transfer across different domains. In this paper, we focus on theUnified Cross-Domain Recommendation(Unified CDR) problem. That is, how to enhance the recommendation performance within and cross domains when users are partially overlapped. It has two main challenges, i.e., 1) how to obtain robust matching solution among the whole users and 2) how to exploit consistent and accurate results across domains. To address these two challenges, we proposeMUCRP, a cross-domain recommendation framework for the Unified CDR problem.MUCRPcontains three modules, i.e., variational rating reconstruction module, robust variational embedding alignment module, and cycle-consistent preference extraction module. To solve the first challenge, we propose fused Gromov-Wasserstein distribution co-clustering optimal transport to obtain more robust matching solution via considering both semantic and structure information. To tackle the second challenge, we propose embedding-consistent and prediction-consistent losses via dual autoencoder framework to achieve consistent results. Our empirical study on Douban and Amazon datasets demonstrates thatMUCRPsignificantly outperforms the state-of-the-art models.

Recommender systems are fundamental information filtering techniques to recommend content or items that meet users’ personalities and potential needs. As a crucial solution to address the difficulty of user identification and unavailability of historical information, session-based recommender systems provide recommendation services that only rely on users’ behaviors in the current session. However, most existing studies are not well-designed for modeling heterogeneous user behaviors and capturing the relationships between them in practical scenarios. To fill this gap, in this article, we propose a novel graph-based method, namely H eterogeneous I nformation C rossing on G raphs (HICG). HICG utilizes multiple types of user behaviors in the sessions to construct heterogeneous graphs, and captures users’ current interests with their long-term preferences by effectively crossing the heterogeneous information on the graphs. In addition, we also propose an enhanced version, named HICG-CL, which incorporates the contrastive learning (CL) technique to enhance item representation ability. By utilizing the item co-occurrence relationships across different sessions, HICG-CL improves the recommendation performance of HICG. We conduct extensive experiments on three real-world recommendation datasets, and the results verify that (i) HICG achieves state-of-the-art performance by utilizing multiple types of behaviors on the heterogeneous graph. (ii) HICG-CL further significantly improves the recommendation performance of HICG by the proposed contrastive learning module.

Federated learning (FL) is a popular distributed machine learning framework in which clients aggregate models' parameters instead of sharing individual data.In FL, clients frequently communicate with the server under limited network bandwidth, raising the communication challenge.Multiple compression methods have been proposed to reduce the transmitted parameters.However, these techniques show that the federated performance degrades significantly with Non-IID (non-identically independently distributed) datasets.To address this issue, we propose an effective method called FedPSE, which solves the efficiency challenge of FL with heterogeneous data.FedPSE compresses the local updates on clients using Top-K sparsification and aggregates these updates on the server by element-wise aggregation.Then clients download the personalized sparse updates from the server to update their individual local models.We then theoretically analyze the convergence of FedPSE under the non-convex setting.Moreover, extensive experiments on four benchmark tasks demonstrate that our FedPSE outperforms the state-of-the-art methods on Non-IID datasets in terms of efficiency and accuracy.

Building a graph neural network (GNN)-based recommender system without violating user privacy proves challenging. Existing methods can be divided into federated GNNs and decentralized GNNs. But both methods have undesirable effects, i.e., low communication efficiency and privacy leakage. This paper proposes DGREC, a novel decentralized GNN for privacy-preserving recommendations, where users can choose to publicize their interactions. It includes three stages, i.e., graph construction, local gradient calculation, and global gradient passing. The first stage builds a local inner-item hypergraph for each user and a global inter-user graph. The second stage models user preference and calculates gradients on each local device. The third stage designs a local differential privacy mechanism named secure gradient-sharing, which proves strong privacy-preserving of users' private data. We conduct extensive experiments on three public datasets to validate the consistent superiority of our framework.

In recommendation scenarios, there are two long-standing challenges, i.e., selection bias and data sparsity, which lead to a significant drop in prediction accuracy for both Click-Through Rate (CTR) and post-click Conversion Rate (CVR) tasks. To cope with these issues, existing works emphasize on leveraging Multi-Task Learning (MTL) frameworks (Category 1) or causal debiasing frameworks (Category 2) to incorporate more auxiliary data in the entire exposure/inference space $\mathcal{D}$ or debias the selection bias in the click/training space ${\mathcal{O}}$. However, these two kinds of solutions cannot effectively address the not-missing-at-random problem and debias the selection bias in ${\mathcal{O}}$ to fit the inference in $\mathcal{D}$. To fill the research gaps, we propose a Direct entire-space Causal Multi-Task framework, namely DCMT, for post-click conversion prediction in this paper. Specifically, inspired by users’ decision process of conversion, we propose a new counterfactual mechanism to debias the selection bias in $\mathcal{D}$, which can predict the factual CVR and the counterfactual CVR under the soft constraint of a counterfactual prior knowledge. Extensive experiments demonstrate that our DCMT can improve the state-of-the-art methods by an average of 1.07% in term of CVR AUC on the offline datasets and 0.75% in term of PV-CVR on the online A/B test (the Alipay Search). Such improvements can increase millions of conversions per week in real industrial applications, e.g., the Alipay Search.

Personalized diagnosis prediction based on electronic health records (EHR) of patients is a promising yet challenging task for AI in healthcare. Existing studies typically ignore the heterogeneity of diseases across different patients. For example, diabetes can have different complications across different patients (e.g., hyperlipidemia and circulatory disorder), which requires personalized diagnoses and treatments. Specifically, existing models fail to consider 1) varying severity of the same diseases for different patients, 2) complex interactions among syndromic diseases, and 3) dynamic progression of chronic diseases. In this work, we propose to perform personalized diagnosis prediction based on EHR data via capturing disease severity, interaction, and progression. In particular, we enable personalized disease representations via severity-driven embeddings at the disease level. Then, at the visit level, we propose to capture higher-order interactions among diseases that can collectively affect patients’ health status via hypergraph-based aggregation; at the patient level, we devise a personalized generative model based on neural ordinary differential equations to capture the continuous-time disease progressions underlying discrete and incomplete visits. Extensive experiments on two real-world EHR datasets show significant performance gains brought by our approach, yielding average improvements of 10.70% for diagnosis prediction over state-of-the-art competitors.

Cross-Domain Recommendation (CDR) aims to solve the data sparsity problem by integrating the strengths of different domains. Though researchers have proposed various CDR methods to effectively transfer knowledge across domains, they fail to address the following key issues, i.e., (1) they cannot model high-order correlations among users and items in every single domain to obtain more accurate representations; (2) they cannot model the correlations among items across different domains. To tackle the above issues, we propose a novel Intra and Inter Domain HyperGraph Convolutional Network (II-HGCN) framework, which includes two main layers in the modeling process, i.e., the intra-domain layer and the inter-domain layer. In the intra-domain layer, we design a user hypergraph and an item hypergraph to model high-order correlations inside every single domain. Thus we can address the data sparsity problem better and learn high-quality representations of users and items. In the inter-domain layer, we propose an inter-domain hypergraph structure to explore correlations among items from different domains based on their interactions with common users. Therefore we can not only transfer the knowledge of users but also combine embeddings of items across domains. Comprehensive experiments on three widely used benchmark datasets demonstrate that II-HGCN outperforms other state-of-the-art methods, especially when datasets are extremely sparse.

Sequential Cross-Domain Recommendation (CDR) has been popularly studied to utilize different domain knowledge and users’ historical behaviors for the next-item prediction. In this paper, we focus on the cross-domain sequential recommendation problem. This commonly exist problem is rather challenging from two perspectives, i.e., the implicit user historical rating sequences are difficult in modeling and the users/items on different domains are mostly non-overlapped. Most previous sequential CDR approaches cannot solve the cross-domain sequential recommendation problem well, since (1) they cannot sufficiently depict the users’ actual preferences, (2) they cannot leverage and transfer useful knowledge across domains. To tackle the above issues, we propose joint Internal multi-interest exploration and External domain alignment for cross domain Sequential Recommendation model (IESRec). IESRec includes two main modules, i.e., internal multi-interest exploration module and external domain alignment module. To reflect the users’ diverse characteristics with multi-interests evolution, we first propose internal temporal optimal transport method in the internal multi-interest exploration module. We further propose external alignment optimal transport method in the external domain alignment module to reduce domain discrepancy for the item embeddings. Our empirical studies on Amazon datasets demonstrate that IESRec significantly outperforms the state-of-the-art models.

Session-based Recommendation aims at predicting the next interacted item based on short anonymous behavior sessions. However, existing solutions neglect to model two inherent properties of sequential representing distributions, i.e., hierarchy structures resulted from item popularity and collaborations existing in both intra- and inter-session. Tackling with these two factors at the same time is challenging. On the one hand, traditional Euclidean space utilized in previous studies fails to capture hierarchy structures due to a restricted representation ability. On the other hand, the intuitive apply of hyperbolic geometry could extract hierarchical patterns but more emphasis on degree distribution weakens intra- and inter-session collaborations. To address the challenges, we propose a Hierarchy-Aware Dual Clustering Graph Network (HADCG) model for session-based recommendation. Towards the first challenge, we design the hierarchy-aware graph modeling module which converts sessions into hyperbolic session graphs, adopting hyperbolic geometry in propagation and attention mechanism so as to integrate chronological and hierarchical information. As for the second challenge, we introduce the deep dual clustering module which develops a two-level clustering strategy, i.e., information regularizer for intra-session clustering and contrastive learner for inter-session clustering, to enhance hyperbolic representation learning from collaborative perspectives and further promote recommendation performance. Extensive experiments on three real-world datasets demonstrate the effectiveness of the proposed HADCG.

Spatio-temporal kriging is an important problem in web and social applications, such as Web or Internet of Things, where things (e.g., sensors) connected into a web often come with spatial and temporal properties. It aims to infer knowledge for (the things at) unobserved locations using the data from (the things at) observed locations during a given time period of interest. This problem essentially requires inductive learning. Once trained, the model should be able to perform kriging for different locations including newly given ones, without retraining. However, it is challenging to perform accurate kriging results because of the heterogeneous spatial relations and diverse temporal patterns. In this paper, we propose a novel inductive graph representation learning model for spatio-temporal kriging. We first encode heterogeneous spatial relations between the unobserved and observed locations by their spatial proximity, functional similarity, and transition probability. Based on each relation, we accurately aggregate the information of most correlated observed locations to produce inductive representations for the unobserved locations, by jointly modeling their similarities and differences. Then, we design relation-aware gated recurrent unit (GRU) networks to adaptively capture the temporal correlations in the generated sequence representations for each relation. Finally, we propose a multi-relation attention mechanism to dynamically fuse the complex spatio-temporal information at different time steps from multiple relations to compute the kriging output. Experimental results on three real-world datasets show that our proposed model outperforms state-of-the-art methods consistently, and the advantage is more significant when there are fewer observed locations. Our code is available at https://github.com/zhengchuanpan/INCREASE.

Cross-Domain Recommendation has been popularly studied to utilize different domain knowledge to solve the cold-start problem in recommender systems. In this paper, we focus on theCross-Domain Cold-Start Recommendation(CDCSR) problem. That is, how to leverage the information from a source domain, where items are ’warm’, to improve the recommendation performance of a target domain, where items are ’cold’. It has two main challenges, i.e., (1) how to efficiently reduce the discrepancy between the latent embedding distribution across domains and (2) how to generate more robust and stable cold item embeddings. To address these two challenges, we proposeCPKSPA, a cross-domain recommendation framework for the CDCSR problem.CPKSPAcontains three modules, i.e., rating prediction module, embedding distribution alignment module, and contrastive augmentation module. To start with, we first utilize the rating prediction module to model user-item interactions. To solve the first challenge, we propose proxy Stein path alignment with typical-subgroup discovering algorithm in the embedding distribution alignment module. To tackle the second challenge, we propose the contrastive augmentation module which adopts contrastive augmentation learning to generate more stable and robust cold item embeddings. Our empirical study on Douban and Amazon datasets demonstrates thatCPKSPAsignificantly outperforms the state-of-the-art models.

The sparse interactions between users and items on the web have aggravated the difficulty of their representations in recommender systems. Existing approaches leverage item attributes (e.g., item categories and tags) to alleviate the data sparsity problem, so as to enhance the performance and interpretability of recommendation. However, directly using all attributes of items cannot avoid the negative impacts of low-quality attributes, where manually labeling the quality of attributes is time-consuming. To this end, we propose HQRec to jointly measure the quality of attributes automatically and perform recommendation accurately. Specifically, we first analyze the different qualities among item attributes, and propose to leverage item categories to select high-quality tags via category-guided quality measurement and direction-aware optimization in an unsupervised fashion. Then, we propose to capture the complex relations among users and items based on the high-quality attributes, where a novel quality-aware embedding fusion and quality-aware embedding propagation mechanism for users and items is devised. Extensive experiments on four real-world benchmark datasets show drastic performance gains brought by our proposed HQRec framework, which constantly achieves an average of 14.73% improvement over the state-of-the-art baselines in terms of Recall and NDCG metrics. Insightful case studies also show that our automatic quality measurements are highly accurate and interpretable.

Cross-Domain Recommendation (CDR) and Cross-System Recommendation (CSR) have been proposed to improve the recommendation accuracy in a target dataset (domain/system) with the help of a source one with relatively richer information. However, most existing CDR and CSR approaches are single-target, namely, there is a single target dataset, which can only help the target dataset and thus cannot benefit the source dataset. In this paper, we focus on three new scenarios, i.e., Dual-Target CDR (DTCDR), Multi-Target CDR (MTCDR), and CDR+CSR, and aim to improve the recommendation accuracy in all datasets simultaneously for all scenarios. To do this, we propose a unified framework, called GA (based onGraph embedding andAttention techniques), for all three scenarios. In GA, we first construct separate heterogeneous graphs to generate more representative user and item embeddings. Then, we propose an element-wise attention mechanism to effectively combine the embeddings of common entities (users/items) learned from different datasets. Moreover, to avoid negative transfer, we further propose aPersonalized training strategy to minimize the embedding difference of common entities between a richer dataset and a sparser dataset, deriving three new models, i.e., GA-DTCDR-P, GA-MTCDR-P, and GA-CDR+CSR-P, for the three scenarios respectively. Extensive experiments conducted on four real-world datasets demonstrate that our proposed GA models significantly outperform the state-of-the-art approaches.

Integrating multiple online social networks (OSNs) has important implications for many downstream social mining tasks, such as user preference modelling, recommendation, and link prediction. However, it is unfortunately accompanied by growing privacy concerns about leaking sensitive user information. How to fully utilize the data from different online social networks while preserving user privacy remains largely unsolved. To this end, we propose a Cross-network Social User Embedding framework, namely DP-CroSUE, to learn the comprehensive representations of users in a privacy-preserving way. We jointly consider information from partially aligned social networks with differential privacy guarantees. In particular, for each heterogeneous social network, we first introduce a hybrid differential privacy notion to capture the variation of privacy expectations for heterogeneous data types. Next, to find user linkages across social networks, we make unsupervised user embedding-based alignment in which the user embeddings are achieved by the heterogeneous network embedding technology. To further enhance user embeddings, a novel cross-network GCN embedding model is designed to transfer knowledge across networks through those aligned users. Extensive experiments on three real-world datasets demonstrate that our approach makes a significant improvement on user interest prediction tasks as well as defending user attribute inference attacks from embedding.

The sparse interactions between users and items on the web have aggravated the difficulty of their representations in recommender systems. Existing approaches leverage tags to alleviate the data sparsity problem, so as to enhance the performance and interpretability of recommendation. However, directly using flat item tags fails to fully exploit the hierarchical relations in data, but tag taxonomies are not always available. To this end, we propose TaxoRec to jointly construct a tag taxonomy automatically and perform recommendation accurately in hyperbolic space. Specifically, we first leverage hyperbolic space and enable the optimization of a discrete taxonomy structure via a representation-aware scoring function and an adaptive clustering algorithm, and preserve the hierarchical structure for interpretability. Then, we propose to capture the complex relations among users, items, and tags in a unified hyperbolic metric space, where a novel tag-enhanced aggregation mechanism and tag-enhanced metric learning algorithm for users and items are defined. Extensive experiments on four real-world benchmark datasets show drastic performance gains brought by our proposed TaxoRec framework, which constantly achieves an average of 7.76% improvement over the state-of-the-art baselines regarding both Recall and NDCG metrics. Insightful case studies also show that our automatically constructed tag taxonomies are highly accurate and interpretable.

Drug recommendation is an important task of AI for healthcare. To recommend proper drugs, existing methods rely on various clinical records (e.g., diagnosis and procedures), which are commonly found in data such as electronic health records (EHRs). However, detailed records as such are often not available and the inputs might merely include a set of symptoms provided by doctors. Moreover, existing drug recommender systems usually treat drugs as individual items, ignoring the unique requirements that drug recommendation has to be done on a set of items (drugs), which should be as small as possible and safe without harmful drug-drug interactions (DDIs).

Deep graph learning (DGL) has achieved remarkable progress in both business and scientific areas ranging from finance and e-commerce, to drug and advanced material discovery. Despite the progress, how to ensure various DGL algorithms behave in a socially responsible manner and meet regulatory compliance requirements becomes an emerging problem, especially in risk-sensitive domains. Trustworthy graph learning (TwGL) aims to solve the above problems from a technical viewpoint. In contrast to conventional graph learning which mainly cares about model performance, TwGL considers various reliability and safety aspects of DGL, including but not limited to adversarial robustness, explainability, and privacy protection. Whilst several previous tutorials have been made for the introduction of DGL in KDD, seldom is there a special focus on its safety aspects, including reliability, explainability, and privacy protection capability. This tutorial mainly covers the key achievements of trustworthy graph learning in recent years. Specifically, we will discuss three essential topics, that is, the reliability of DGL against inherent noise, distribution shift and adversarial attack, explainability methods, and privacy protection for DGL. Meanwhile, we will introduce some guidelines for applying DGL to risk-sensitive applications (e.g., AI drug discovery) to ensure GNN models behave in a trustworthy way. We hope our tutorial can offer a comprehensive review of recent advances in this area and also provide some useful suggestions to guide the developers to choose appropriate techniques for their applications.

Cross-Domain Recommendation (CDR) has been popularly studied to utilize different domain knowledge to solve the cold-start problem in recommender systems. Most of the existing CDR models assume that both the source and target domains share the same overlapped user set for knowledge transfer. However, only few proportion of users simultaneously activate on both the source and target domains in practical CDR tasks. In this paper, we focus on the Partially Overlapped Cross-Domain Recommendation (POCDR) problem, that is, how to leverage the information of both the overlapped and non-overlapped users to improve recommendation performance. Existing approaches cannot fully utilize the useful knowledge behind the non-overlapped users across domains, which limits the model performance when the majority of users turn out to be non-overlapped. To address this issue, we propose an end-to-end Dual-autoencoder with Variational Domain-invariant Embedding Alignment (VDEA) model, a cross-domain recommendation framework for the POCDR problem, which utilizes dual variational autoencoders with both local and global embedding alignment for exploiting domain-invariant user embedding. VDEA first adopts variational inference to capture collaborative user preferences, and then utilizes Gromov-Wasserstein distribution co-clustering optimal transport to cluster the users with similar rating interaction behaviors. Our empirical studies on Douban and Amazon datasets demonstrate that VDEA significantly outperforms the state-of-the-art models, especially under the POCDR setting.

Cold-start diagnosis prediction is a challenging task for AI in healthcare, where often only a few visits per patient and a few observations per disease can be exploited. Although meta-learning is widely adopted to address the data sparsity problem in general domains, directly applying it to healthcare data is less effective, since it is unclear how to capture both the temporal relations in clinical visits and the complicated relations among syndromic diseases for precise personalized diagnosis. To this end, we first propose a novel Meta-learning framework for cold-start diagnosis prediction in healthCare data (MetaCare). By explicitly encoding the effects of disease progress over time as a generalization prior, MetaCare dynamically predicts future diagnosis and timestamp for infrequent patients. Then, to model complicated relations among rare diseases, we propose to utilize domain knowledge of hierarchical relations among diseases, and further perform diagnosis subtyping to mine the latent syndromic relations among diseases. Finally, to tailor the generic meta-learning framework with personalized parameters, we design a hierarchical patient subtyping mechanism and bridge the modeling of both infrequent patients and rare diseases. We term the joint model as MetaCare++. Extensive experiments on two real-world benchmark datasets show significant performance gains brought by MetaCare++, yielding average improvements of 7.71% for diagnosis prediction and 13.94% for diagnosis time prediction over the state-of-the-art baselines.

Cross Domain Recommendation (CDR) has been popularly studied to alleviate the cold-start and data sparsity problem commonly existed in recommender systems. CDR models can improve the recommendation performance of a target domain by leveraging the data of other source domains. However, most existing CDR models assume information can directly ‘transfer across the bridge’, ignoring the privacy issues. To solve this problem, we propose a novel two stage based privacy-preserving CDR framework (PriCDR). In the first stage, we propose two methods, i.e., Johnson-Lindenstrauss Transform (JLT) and Sparse-aware JLT (SJLT), to publish the rating matrix of the source domain using Differential Privacy (DP). We theoretically analyze the privacy and utility of our proposed DP based rating publishing methods. In the second stage, we propose a novel heterogeneous CDR model (HeteroCDR), which uses deep auto-encoder and deep neural network to model the published source rating matrix and target rating matrix respectively. To this end, PriCDR can not only protect the data privacy of the source domain, but also alleviate the data sparsity of the source domain. We conduct experiments on two benchmark datasets and the results demonstrate the effectiveness of PriCDR and HeteroCDR.

Cross-Domain Recommendation (CDR) has been popularly studied to utilize different domain knowledge to solve the data sparsity and cold-start problem in recommender systems. In this paper, we focus on the Review-based Non-overlapped Recommendation (RNCDR) problem. The problem is commonly-existed and challenging due to two main aspects, i.e, there are only positive user-item ratings on the target domain and there is no overlapped user across different domains. Most previous CDR approaches cannot solve the RNCDR problem well, since (1) they cannot effectively combine review with other information (e.g., ID or ratings) to obtain expressive user or item embedding, (2) they cannot reduce the domain discrepancy on users and items. To fill this gap, we propose Collaborative Filtering with Attribution Alignment model (CFAA), a cross-domain recommendation framework for the RNCDR problem. CFAA includes two main modules, i.e., rating prediction module and embedding attribution alignment module. The former aims to jointly mine review, one-hot ID, and multi-hot historical ratings to generate expressive user and item embeddings. The later includes vertical attribution alignment and horizontal attribution alignment, tending to reduce the discrepancy based on multiple perspectives. Our empirical study on Douban and Amazon datasets demonstrates that CFAA significantly outperforms the state-of-the-art models under the RNCDR setting.

Deep Neural Networks (DNNs) have achieved remarkable progress in various real-world applications, especially when abundant training data are provided. However, data isolation has become a serious problem currently. Existing works build privacy-preserving DNN models from either algorithmic perspective or cryptographic perspective. The former mainly splits the DNN computation graph between data holders or between data holders and server, which demonstrates good scalability but suffers from accuracy loss and potential privacy risks. In contrast, the latter leverages time-consuming cryptographic techniques, which has strong privacy guarantee but poor scalability. In this article, we propose SPNN—a Scalable and Privacy-preserving deep Neural Network learning framework, from an algorithmic-cryptographic co-perspective. From algorithmic perspective, we split the computation graph of DNN models into two parts, i.e., the private-data-related computations that are performed by data holders and the rest heavy computations that are delegated to a semi-honest server with high computation ability. From cryptographic perspective, we propose using two types of cryptographic techniques, i.e., secret sharing and homomorphic encryption, for the isolated data holders to conduct private-data-related computations privately and cooperatively. Furthermore, we implement SPNN in a decentralized setting and introduce user-friendly APIs. Experimental results conducted on real-world datasets demonstrate the superiority of our proposed SPNN.

Privacy-preserving machine learning has drawn increasingly attention recently, especially with kinds of privacy regulations come into force. Under such situation, Federated Learning (FL) appears to facilitate privacy-preserving joint modeling among multiple parties. Although many federated algorithms have been extensively studied, there is still a lack of secure and practical gradient tree boosting models (e.g., XGB) in literature. In this paper, we aim to build large-scale secure XGB under vertically federated learning setting. We guarantee data privacy from three aspects. Specifically, (1) we employ secure multi-party computation techniques to avoid leaking intermediate information during training, (2) we store the output model in a distributed manner in order to minimize information release, and (3) we provide a novel algorithm for secure XGB predict with the distributed model. Furthermore, by proposing secure permutation protocols, we can improve the training efficiency and make the framework scale to large dataset. We conduct extensive experiments on both public datasets and real-world datasets, and the results demonstrate that our proposed XGB models provide not only competitive accuracy but also practical performance.

Logistic Regression (LR) is the most widely used machine learning model in industry for its efficiency, robustness, and interpretability. Due to the problem of data isolation and the requirement of high model performance, many applications in industry call for building a secure and efficient LR model for multiple parties. Most existing work uses either Homomorphic Encryption (HE) or Secret Sharing (SS) to build secure LR. HE based methods can deal with high-dimensional sparse features, but they incur potential security risks. SS based methods have provable security, but they have efficiency issue under high-dimensional sparse features. In this paper, we first present CAESAR, which combines HE and SS to build secure large-scale sparse logistic regression model and achieves both efficiency and security. We then present the distributed implementation of CAESAR for scalability requirement. We have deployed CAESAR in a risk control task and conducted comprehensive experiments. Our experimental results show that CAESAR improves the state-of-the-art model by around 130 times.

With the rapid growth of data volume, data-driven machine learning models have become a necessary part of many industrial applications. Intuitively, the more high-quality data used for training leads to better model performance. However, in reality, data are usually scattered and isolated in different organizations or companies. Such a "data isolation" problem stimulates both academia and industry to explore the collaborative learning paradigm to build better models jointly with multiple data sources. Despite the potential performance gains, this learning paradigm inevitably faces privacy issues, especially for the Fintech domain where data are sensitive by nature. In this paper, we present a privacy-preserving collaborative learning system in Ant Financial, named Nebula. Our system aims to facilitate privacy-preserving collaborative model training for industrial-scale applications. Our system is built upon a ring-allreduce MPI based distributed framework. On top of that, with some optimization strategies and novel sharing scheme, our system is able to scale up to tens of millions of data samples with hundreds of thousands of features and achieve more than 100x speedup compared with the existing state-of-the-art implementations.

Point-of-Interest (POI) recommendation has been extensively studied and successfully applied in industry recently. However, most existing approaches build centralized models on the basis of collecting users’ data. Both private data and models are held by the recommender, which causes serious privacy concerns. In this article, we propose a novel Privacy preserving POI Recommendation (PriRec) framework. First, to protect data privacy, users’ private data (features and actions) are kept on their own side, e.g., Cellphone or Pad. Meanwhile, the public data that need to be accessed by all the users are kept by the recommender to reduce the storage costs of users’ devices. Those public data include: (1) static data only related to the status of POI, such as POI categories, and (2) dynamic data dependent on user-POI actions such as visited counts. The dynamic data could be sensitive, and we develop local differential privacy techniques to release such data to the public with privacy guarantees. Second, PriRec follows the representations of Factorization Machine (FM) that consists of a linear model and the feature interaction model. To protect the model privacy, the linear models are saved on the users’ side, and we propose a secure decentralized gradient descent protocol for users to learn it collaboratively. The feature interaction model is kept by the recommender since there is no privacy risk, and we adopt a secure aggregation strategy in a federated learning paradigm to learn it. To this end, PriRec keeps users’ private raw data and models in users’ own hands, and protects user privacy to a large extent. We apply PriRec in real-world datasets, and comprehensive experiments demonstrate that, compared with FM, PriRec achieves comparable or even better recommendation accuracy.

Secure online transaction is an essential task for e-commerce platforms. Alipay, one of the world’s leading cashless payment platform, provides the payment service to both merchants and individual customers. The fraud detection models are built to protect the customers, but stronger demands are raised by the new scenes, which are lacking in training data and labels. The proposed model makes a difference by utilizing the data under similar old scenes and the data under a new scene is treated as the target domain to be promoted. Inspired by this real case in Alipay, we view the problem as a transfer learning problem and design a set of revise strategies to transfer the source domain models to the target domain under the framework of gradient boosting tree models. This work provides an option for the cold-start and data-sharing problems.

In order to address the data sparsity problem in recommender systems, in recent years, Cross-Domain Recommendation (CDR) leverages the relatively richer information from a source domain to improve the recommendation performance on a target domain with sparser information. However, each of the two domains may be relatively richer in certain types of information (e.g., ratings, reviews, user profiles, item details, and tags), and thus, if we can leverage such information well, it is possible to improve the recommendation performance on both domains simultaneously (i.e., dual-target CDR), rather than a single target domain only. To this end, in this paper, we propose a new framework, DTCDR, for Dual-Target Cross-Domain Recommendation. In DTCDR, we first extensively utilize rating and multi-source content information to generate rating and document embeddings of users and items. Then, based on Multi-Task Learning (MTL), we design an adaptable embedding-sharing strategy to combine and share the embeddings of common users across domains, with which DTCDR can improve the recommendation performance on both richer and sparser (i.e., dual-target) domains simultaneously. Extensive experiments conducted on real-world datasets demonstrate that DTCDR can significantly improve the recommendation accuracies on both richer and sparser domains and outperform the state-of-the-art single-domain and cross-domain approaches.

Internet companies are facing the need for handling large-scale machine learning applications on a daily basis and distributed implementation of machine learning algorithms which can handle extra-large-scale tasks with great performance is widely needed. Deep forest is a recently proposed deep learning framework which uses tree ensembles as its building blocks and it has achieved highly competitive results on various domains of tasks. However, it has not been tested on extremely large-scale tasks. In this work, based on our parameter server system, we developed the distributed version of deep forest. To meet the need for real-world tasks, many improvements are introduced to the original deep forest model, including MART (Multiple Additive Regression Tree) as base learners for efficiency and effectiveness consideration, the cost-based method for handling prevalent class-imbalanced data, MART based feature selection for high dimension data, and different evaluation metrics for automatically determining the cascade level. We tested the deep forest model on an extra-large-scale task, i.e., automatic detection of cash-out fraud, with more than 100 million training samples. Experimental results showed that the deep forest model has the best performance according to the evaluation metrics from different perspectives even with very little effort for parameter tuning. This model can block fraud transactions in a large amount of money each day. Even compared with the best-deployed model, the deep forest model can additionally bring a significant decrease in economic loss each day.

We present, GEM, the first heterogeneous graph neural network approach for detecting malicious accounts at Alipay, one of the world's leading mobile cashless payment platform. Our approach, inspired from a connected subgraph approach, adaptively learns discriminative embeddings from heterogeneous account-device graphs based on two fundamental weaknesses of attackers, i.e. device aggregation and activity aggregation. For the heterogeneous graph consists of various types of nodes, we propose an attention mechanism to learn the importance of different types of nodes, while using the sum operator for modeling the aggregation patterns of nodes in each type. Experiments show that our approaches consistently perform promising results compared with competitive methods over time.

Collaborative filtering, especially latent factor model, has been popularly used in personalized recommendation. Latent factor model aims to learn user and item latent factors from user-item historic behaviors. To apply it into real big data scenarios, efficiency becomes the first concern, including offline model training efficiency and online recommendation efficiency. In this paper, we propose a D istributed C ollaborative H ashing ( DCH ) model which can significantly improve both efficiencies. Specifically, we first propose a distributed learning framework, following the state-of-the-art parameter server paradigm, to learn the offline collaborative model. Our model can be learnt efficiently by distributedly computing subgradients in minibatches on workers and updating model parameters on servers asynchronously. We then adopt hashing technique to speedup the online recommendation procedure. Recommendation can be quickly made through exploiting lookup hash tables. We conduct thorough experiments on two real large-scale datasets. The experimental results demonstrate that, comparing with the classic and state-of-the-art (distributed) latent factor models, DCH has comparable performance in terms of recommendation accuracy but has both fast convergence speed in offline model training procedure and realtime efficiency in online recommendation procedure. Furthermore, the encouraging performance of DCH is also shown for several real-world applications in Ant Financial.

Recently, latent factor model (LFM) has been drawing much attention in recommender systems due to its good performance and scalability. However, existing LFMs predict missing values in a user-item rating matrix only based on the known ones, and thus the sparsity of the rating matrix always limits their performance. Meanwhile, semi-supervised learning (SSL) provides an effective way to alleviate the label (i.e., rating) sparsity problem by performing label propagation, which is mainly based on the smoothness insight on affinity graphs. However, graph-based SSL suffers serious scalability and graph unreliable problems when directly being applied to do recommendation. In this article, we propose a novel probabilistic chain graph model (CGM) to marry SSL with LFM. The proposed CGM is a combination of Bayesian network and Markov random field . The Bayesian network is used to model the rating generation and regression procedures, and the Markov random field is used to model the confidence-aware smoothness constraint between the generated ratings. Experimental results show that our proposed CGM significantly outperforms the state-of-the-art approaches in terms of four evaluation metrics, and with a larger performance margin when data sparsity increases.

Nowadays, the number of mobile applications (Apps) is rapidly increasing due to different kinds of user needs. Existing App recommendation models can not scale to practical large data scenarios. In this paper, we present a large scale App recommendation framework, which has been applied into real application in Ant Financial for recommending Apps for hundreds of millions of users. We first present feature engineering, including a novel way to high-level features, and then use a linear ranking model to make recommendation. We also report the encouraging results through the experiments in real large scale dataset.

In recent years, due to the emergence of Big Data (terabytes or petabytes) and Big Model (tens of billions of parameters), there has been an ever-increasing need of parallelizing machine learning (ML) algorithms in both academia and industry. Although there are some existing distributed computing systems, such as Hadoop and Spark, for parallelizing ML algorithms, they only provide synchronous and coarse-grained operators (e.g., Map, Reduce, and Join, etc.), which may hinder developers from implementing more efficient algorithms. This motivated us to design a universal distributed platform termed KunPeng, that combines both distributed systems and parallel optimization algorithms to deal with the complexities that arise from large-scale ML. Specifically, KunPeng not only encapsulates the characteristics of data/model parallelism, load balancing, model sync-up, sparse representation, industrial fault-tolerance, etc., but also provides easy-to-use interface to empower users to focus on the core ML logics. Empirical results on terabytes of real datasets with billions of samples and features demonstrate that, such a design brings compelling performance improvements on ML programs ranging from Follow-the-Regularized-Leader Proximal algorithm to Sparse Logistic Regression and Multiple Additive Regression Trees. Furthermore, KunPeng's encouraging performance is also shown for several real-world applications including the Alibaba's Double 11 Online Shopping Festival and Ant Financial's transaction risk estimation.