As a powerful tool to alleviate the data sparsity and cold-start problems in traditional recommender systems, cross-domain recommendation hinges on addressing two fundamental questions: how to transfer knowledge and what to transfer. Regarding the two questions, existing methods have limitations, such as restricted domain connections, inadequate representation disentanglement, and insufficient knowledge transfer. To overcome these challenges, we propose a novel model, KDCLM, which integrates sophisticated knowledge distillation and contrastive learning mechanisms within a multi-view graph architecture. The proposed model comprises two views—a local view and a global view—both of which construct multiple graphs based on user–item interactions to establish richer domain connections. Specifically, the local view incorporates two contrastive learning mechanisms: one for aligning domain-invariant representations and another for differentiating domain-specific representations, which jointly achieve effective representation disentanglement. In addition, we employ knowledge distillation between the global heterogeneous user–item interaction graph and the homogeneous user–user and item–item relationship graphs to facilitate sufficient knowledge transfer. Through extensive experiments on real-world cross-domain recommendation tasks, our proposed KDCLM model demonstrates significant improvements over current state-of-the-art methods. We release our source code at https://github.com/fanydan/KDCLM .

As necessary information about whether cooperation can be reached, rewards should be determined in advance in Vertical Federated Learning (VFL). To determine reasonable rewards, participant contributions should be estimated precisely. We propose a Vertically Federated Contribution Estimation (VF-CE) method. VF-CE calculates Mutual Information (MI) between distributed features and the label using a neural network trained via VFL itself. Note that compensation for CE is low as it only covers computation costs, and reward for real VFL training is high as it needs to cover training costs as well as participants' contributions to model performance and the resulting business benefits. Because MI presents a strong positive correlation with the final model performance, contributions to model performance can be estimated based on contributions to MI. We integrate a scalar-level attention mechanism in MI neural network. The attention weights of participants are treated as their contributions. We find that attention weights can effectively measure contribution redundancy, as its Spearman correlation coefficient with Shapley value is as high as 0.963. We demonstrate that VF-CE also satisfies properties of balance, zero element, and symmetry concerning fairness, which are hallmark properties of Shapley value. Compared with existing work, we consider contribution redundancy precisely, efficiently output approximated Shapley values through one MI calculation instead of 2 n where n is the number of participants, and introduce no extra privacy risk except the inherent risk in VFL, i.e., gradient transmission.

Deep learning-based recommender systems have achieved remarkable success in recent years. However, these methods usually heavily rely on labeled data (i.e., user-item interactions), suffering from problems such as data sparsity and cold-start. Self-supervised learning, an emerging paradigm that extracts information from unlabeled data, provides insights into addressing these problems. Specifically, contrastive self-supervised learning, due to its flexibility and promising performance, has attracted considerable interest and recently become a dominant branch in self-supervised learning-based recommendation methods. In this survey, we provide an up-to-date and comprehensive review of current contrastive self-supervised learning-based recommendation methods. Firstly, we propose a unified framework for these methods. We then introduce a taxonomy based on the key components of the framework, including view generation strategy, contrastive task, and contrastive objective. For each component, we provide detailed descriptions and discussions to guide the choice of the appropriate method. Finally, we outline open issues and promising directions for future research.

Review-based recommender systems explore semantic aspects of users’ preferences by incorporating user-generated reviews into rating-based models. Recent works have demonstrated the potential of review information to improve the recommendation capacity. However, most existing studies rely on optimizing review-based representation learning part, thus failing to explicitly capture the fine-grained semantic aspects, and also ignoring the intrinsic correlation between ratings and reviews. To address these problems, we propose a multi-aspect graph contrastive learning framework, named MAGCL, with three distinctive designs: (i) a multi-aspect representation learning module, which projects semantic relations to different subspaces by decoupling review information, and then obtains high-order decoupled representations in each aspect via graph encoder. (ii) the contrastive learning module performs graph contrastive learning to capture the correlation between rating and review patterns, which utilize unlabeled data to generate self-supervised signals and, in turn, relieve the data sparsity problem of supervision signals. (iii) the multi-task learning module conducts joint training to learn high-order structure-aware yet self-discriminative node representations by combining recommendation task and self-supervised task, which helps alleviate the over-smoothing problem. Extensive experiments are conducted on four real-world review datasets and the results show the superiority of the proposed framework MAGCL compared with several state of the arts. We also provide further analysis on multi-aspect representations and graph contrastive learning to verify the advantage of proposed framework.

Cross-domain recommendation (CDR), which leverages information collected from other domains, has been empirically demonstrated to effectively alleviate data sparsity and cold-start problems encountered in traditional recommendation systems. However, current CDR methods, including those considering time information, do not jointly model the general and current interests within and across domains, which is pivotal for accurately predicting users’ future interactions. In this article, we propose a Contrastive learning-enhanced Multi-View interest learning model (CMVCDR) for cross-domain sequential recommendation. Specifically, we design a static view and a sequential view to model uses’ general interests and current interests, respectively. We divide a user’s general interest representation into a domain-invariant part and a domain-specific part. A cross-domain contrastive learning objective is introduced to impose constraints for optimizing these representations. In the sequential view, we first devise an attention mechanism guided by users’ domain-invariant interest representations to distill cross-domain knowledge pertaining to domain-invariant factors while reducing noise from irrelevant factors. We further design a domain-specific interest-guided temporal information aggregation mechanism to generate users’ current interest representations. Extensive experiments demonstrate the effectiveness of our proposed model compared with state-of-the-art methods.

Graph Convolutional Networks (GCNs) have been widely used for collaborative filtering, due to their effectiveness in exploiting high-order collaborative signals. However, two issues have not been well addressed by existing studies. First, usually only one kind of information is utilized, i.e., user preference in user-item graphs or item dependency in item-item graphs. Second, they usually adopt static graphs, which cannot retain the temporal evolution of the information. These can limit the recommendation quality. To address these limitations, we propose to mine three kinds of information (user preference, item dependency, and user behavior similarity) and their temporal evolution by constructing multiple discrete dynamic heterogeneous graphs (i.e., a user-item dynamic graph, an item-item dynamic graph, and a user-subseq dynamic graph) from interaction data. A novel network (PDGCN) is proposed to learn the representations of users and items in these dynamic graphs. Moreover, we designed a structural neighbor aggregation module with novel pooling and convolution operations to aggregate the features of structural neighbors. We also design a temporal neighbor aggregation module based on self-attention mechanism to aggregate the features of temporal neighbors. We conduct extensive experiments on four real-world datasets. The results indicate that our approach outperforms several competing methods in terms of Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG). Dynamic graphs are also shown to be effective in improving recommendation performance.

Textual reviews contain rich semantic information that is useful for making better recommendation, as such semantic information may indicate more fine-grained preferences of users. Recent efforts make considerable improvement on recommendation by integrating textual reviews in rating-based recommendations. However, there still exist major challenges on integrating textual reviews for recommendation. On the one hand, most existing works focus on learning a single representation from reviews but ignoring complex relations between users (or items) and reviews, which may fail to capture user preferences and item attributes together. On the other hand, these works independently learn latent representations from ratings and reviews while omitting correlations between rating-based features and review-based features, which may harm recommendation performance. In this article, we capture the aspect-aware relations by constructing heterogeneous graphs from reviews. Furthermore, we propose a new recommendation model, namely AHOR, to jointly distill rating-based features and review-based features, which are derived from ratings and reviews, respectively. To explore the multi-hop connectivity information between users, items, and aspects, a novel graph neural network is introduced to learn aspect-aware high-order representations. Experiments based on public datasets show that our approach outperforms state-of-the-art methods. We also provide detailed analysis on the high-order signals and the aspect importance to show the interpretability of our proposed model.

User cold-start scenarios pose great challenges to recommendation systems in accurately capturing user preferences with sparse interaction records. Besides incorporating auxiliary information to enrich user/item representations, recent studies under the schema of meta-learning focus on quickly adapting personalized recommendation models based on cold-start users’ scarce interactions. The majority of meta-learning based recommendation methods follow a bi-level optimization paradigm and learn globally shared initialization across all cold-start recommendation tasks. In addition, to further facilitate the ability of fast adaptation, existing methods have made efforts to tailor task-specific prior knowledge by identifying the individual characteristics of each task. However, we argue that multi-view commonalities between existing users and cold-start users are also essential for precisely distinguishing new tasks, but not comprehensively modeled in previous studies. In this article, we propose a multifaceted relation-aware meta-learning approach namely MeCM for user cold-start recommendation, which enhances task-adaptive initialization customization by extracting multiple views of task relevance. We design a dual customization framework consisting of two successive phases including cluster-level customization and task-level customization. Specifically, MeCM first extracts multifaceted semantic relations between tasks and refines task commonalities into task clusters maintained with memory networks (MNs). Globally learned fast weights corresponding to task clusters are queried to perform cluster-level customization. Then task-level customization is triggered based on contextual information of the target task via interaction-wise encoding. Extensive experiments on real-world datasets demonstrate the superior performance of our model over state-of-the-art meta-learning-based recommendation methods.

Traditional recommendation systems are faced with two long-standing obstacles, namely data sparsity and cold-start problems, which promote the emergence and development of Cross-Domain Recommendation (CDR). The core idea of CDR is to leverage information collected from other domains to alleviate the two problems in one domain. Since the early 2010s, many efforts have been engaged for cross-domain recommendation. Recently, with the development of deep learning and neural networks, a large number of methods have emerged. However, there is a limited number of systematic surveys on CDR, especially regarding the latest proposed methods as well as the recommendation scenarios and recommendation tasks they address. In this survey article, we first proposed a two-level taxonomy of cross-domain recommendation that classifies different recommendation scenarios and recommendation tasks. We then introduce and summarize existing cross-domain recommendation approaches under different recommendation scenarios in a structured manner. We also organize datasets commonly used. We conclude this survey by providing several potential research directions about this field.

Prediction tasks about students have practical significance for both student and college. Making multiple predictions about students is an important part of a smart campus. For instance, predicting whether a student will fail to graduate can alert the student affairs office to take predictive measures to help the student improve his/her academic performance. With the development of information technology in colleges, we can collect digital footprints that encode heterogeneous behaviors continuously. In this article, we focus on modeling heterogeneous behaviors and making multiple predictions together, since some prediction tasks are related and learning the model for a specific task may have the data sparsity problem. To this end, we propose a variant of Long-Short Term Memory (LSTM) and a soft-attention mechanism. The proposed LSTM is able to learn the student profile-aware representation from heterogeneous behavior sequences. The proposed soft-attention mechanism can dynamically learn different importance degrees of different days for every student. In this way, heterogeneous behaviors can be well modeled. In order to model interactions among multiple prediction tasks, we propose a co-attention mechanism based unit. With the help of the stacked units, we can explicitly control the knowledge transfer among multiple tasks. We design three motivating behavior prediction tasks based on a real-world dataset collected from a college. Qualitative and quantitative experiments on the three prediction tasks have demonstrated the effectiveness of our model.

Sequential recommendation which aims to predict a user's next interaction based on his/her previous behaviors, has attracted great attention. Recent studies mainly employ deep recurrent neural networks or self-attention networks to capture dynamic user preferences. However, existing methods merely focus on modeling users' clear interests in interacted items. We argue that for an interaction, the user may also have ambiguous interests in items that are semantically related to the interacted one. For comprehensively capturing user preferences, it is beneficial to discover potential interests from historical interactions at a broader itemset level. Therefore, in this paper, we propose a knowledge graph enhanced sequential recommendation model namely KGIE, which focuses on enhancing user interest modeling with knowledge-enriched itemsets by incorporating the knowledge graph. Specifically, in addition to item-level interest modeling with interacted items, we further construct knowledge-enriched itemsets that are extracted via high-order knowledge associations with the interacted items. For capturing personalized itemset-level interests, we design an attentive aggregation unit to combine item embeddings considering both inherent and contextual personalization signals. Furthermore, to balance the contributions of both two levels of interest modeling, we adaptively learn high-level preference representations with a gating fusion unit. Extensive experiments on three real-world datasets demonstrate the superior performance beyond state-of-the-art methods and recommendation interpretability of our model.

Air pollution is an important environmental issue of increasing concern, which impacts human health. Accurate air quality prediction is crucial for avoiding people suffering from serious air pollution. Most of the prior works focus on capturing the temporal trend of air quality for each monitoring station. Recent deep learning based methods also model spatial dependencies among neighboring stations. However, we observe that besides geospatially adjacent stations, the stations which share similar functionalities or consistent temporal patterns could also have strong dependencies. In this paper, we propose an Attentive Temporal Graph Convolutional Network (ATGCN) to model diverse inter-station relationships for air quality prediction of citywide stations. Specifically, we first encode three types of relationships among stations including spatial adjacency, functional similarity, and temporal pattern similarity into graphs. Then we design parallel encoding modules, which respectively incorporate attentive graph convolution operations into the Gated Recurrent Units (GRUs) to iteratively aggregate features from related stations with different graphs. Furthermore, augmented with an attention-based fusion unit, decoding modules with a similar structure to the encoding modules are designed to generate multi-step predictions for all stations. The experiments on two real-world datasets demonstrate the superior performance of our model beyond state-of-the-art methods.

Crowd flow prediction is a vital problem for an intelligent transportation system construction in a smart city. It plays a crucial role in traffic management and behavioral analysis, thus it has raised great attention from many researchers. However, predicting crowd flows timely and accurately is a challenging task that is affected by many complex factors such as the dependencies of adjacent regions or recent crowd flows. Existing models mainly focus on capturing such dependencies in spatial or temporal domains and fail to model relations between crowd flows of distant regions. We notice that each region has a relatively fixed daily flow and some regions (even very far away from each other) may share similar flow patterns which show strong correlations among them. In this article, we propose a novel model named Double-Encoder which follows a general encoder–decoder framework for multi-step citywide crowd flow prediction. The model consists of two encoder modules named ST-Encoder and FR-Encoder to model spatial-temporal dependencies and daily flow correlations, respectively. We conduct extensive experiments on two real-world datasets to evaluate the performance of the proposed model and show that our model consistently outperforms state-of-the-art methods.

For the emerging mobility-on-demand services, it is of great significance to predict passenger demands based on historical mobility trips towards better vehicle distribution. Prior works have focused on predicting next-step passenger demands at selected locations or hotspots. However, we argue that multi-step citywide passenger demands encapsulate both time-varying demand trends and global statuses, and hence are more beneficial to avoiding demand-service mismatching and developing effective vehicle distribution/scheduling strategies. Furthermore, we find that adaptations of single-step methods are unable to achieve robust prediction with high accuracy for further steps. In this paper, we propose an end-to-end deep neural network model to the prediction task. We employ an encoder-decoder framework based on convolutional and ConvLSTM units to identify complex features that capture spatiotemporal influence and pickup-dropoff interactions on citywide passenger demands. We introduce a multi-level attention model (global attention and temporal attention) to emphasize the effects of latent citywide mobility regularities and capture relevant temporal dependencies. We evaluate our proposed method using real-world mobility trips (taxis and bikes) and the experimental results show that our method achieves higher prediction accuracy than the state-of-the-art approaches.

Prediction tasks about students such as predicting students' academic performances have practical real-world significance at both the student level and the college level. With the rapid construction of smart campuses, colleges not only offer residence and academic programs but also record students' daily life. The digital footprints provide an opportunity to offer better solutions for prediction tasks. In this paper, we aim to propose a general deep neural network which can jointly model student heterogeneous daily behaviors generated from digital footprints and social influence to deal with prediction tasks. To this end, we design a variant of LSTM and a novel attention mechanism to model the daily behavior sequence. The proposed LSTM is able to consider context information (e.g., weather conditions) while modeling the daily behavior sequence. The proposed attention mechanism can dynamically learn the different importance degrees of different days for every student. Based on behavior information, we propose an unsupervised way to construct a social network to model social influence. Moreover, we design a residual network based decoder to model the complex interactions between the features and get the predicted values such as future academic performances. Qualitative and quantitative experiments on two real-world datasets collected from a college have demonstrated the effectiveness of our model.