Sequential recommenders aim to enhance prediction accuracy by leveraging user interaction sequences, with transformer-based models showing particularly strong performance. Among them, cold-start sequential recommenders are particularly challenging because these models typically require extensive historical data to perform optimally. Some works attempt to address this issue by enhancing the adaptive ability of the sequence recommenders with meta-learning approaches. However, they are unsuitable for enhancing the popular Transformer-based sequence recommenders: MAML-based models cannot adapt the large number of parameters of Transformers, while transition-based and metric-based meta-learning models rely on unique architectures that are incompatible with Transformer-based frameworks. Also, they usually lack mechanisms to recognize and cater to multiple interests within short interaction sequences. To address these limitations, we propose MESA, a meta-modulation plugin module specifically designed to enhance the cold-start recommendation of Transformer-based sequential recommender systems. (1) We design a meta-modulation method to directly modulate the parameters in Transformer-based sequence encoders, thus enabling the model to adapt more effectively to new users in cold-start scenarios. (2) Additionally, MESA integrates the Mixture of Experts (MoE) mechanism, which refines sequence representations by utilizing multiple experts, each focusing on different aspects of user interests. This structure enhances the personalization of the recommendation by effectively handling diverse user interests within the sequences. Experiments demonstrate the effectiveness of MESA in cold-start scenarios. Our codes are available here : https://github.com/Mushroom-cat/MESA .

Continuous category discovery (CCD) aims to automatically discover novel categories in continuously arriving unlabeled data. This is a challenging problem considering that there is no number of categories and labels in the newly arrived data, while also needing to mitigate catastrophic forgetting. Most CCD methods cannot handle the contradiction between novel class discovery and classification well. They are also prone to accumulate errors in the process of gradually discovering novel classes. Moreover, most of them use knowledge distillation and data replay to prevent forgetting, occupying more storage space. To address these limitations, we propose Independence-based Diversity and Orthogonality-based Discrimination (IDOD). IDOD mainly includes independent enrichment of diversity module, joint discovery of novelty module, and continuous increment by orthogonality module. In independent enrichment, the backbone is trained separately using contrastive loss to avoid it focusing only on features for classification. Joint discovery transforms multi-stage novel class discovery into single-stage, reducing error accumulation impact. Continuous increment by orthogonality module generates mutually orthogonal prototypes for classification and prevents forgetting with lower space overhead via representative representation replay. Experimental results show that on challenging fine-grained datasets, our method outperforms the state-of-the-art methods.

User cold-start recommendation presents a significant challenge for recommender systems, affecting their overall effectiveness. Meta-learning-based methods have been introduced to address this issue. These methods treat the user cold-start recommendation problem as a few-shot learning task, where each user represents a unique task. The objective is to acquire shared initialization parameters that can be effectively applied across all cold-start users. Subsequently, these shared parameters are fine-tuned into personalized parameters using individual interaction data. Recent studies argue that shared parameters are unsuitable for all users with an implicit grouping distribution of user preference. Therefore, they propose adaptive-initialization-based methods, which first differentiate tasks based on user preferences and then generate task-adaptive initialization parameters using task representations. However, both the meta-learning and adaptive-initialization-based manners ignore discovering the adaptive capability of update strategies in the process of transferring initialization parameters to personalized parameters. Instead, they rely on task-shared optimization strategies, leading the model to fall into an overfitting or underfitting situation. In response to this, we propose a dual-adaptive update strategies-enhanced meta-optimization framework (DAUS) for user cold-start recommendation. First, we integrate dual-adaptive update strategies to enhance the adaptive capability of transferring initialization parameters. This involves incorporating both task-adaptive optimization hyperparameters and objectives. Second, we design a multifaceted task encoder , which can provide diverse task information to differentiate between tasks, including explicit task features (task relevance, training signals) and other implicit task information. Extensive experiments based on three real-world datasets demonstrate that our DAUS outperforms the state-of-the-art methods. The source code is available at https://github.com/XuHao-bit/DAUS .

Sequential recommendation is a critical part of the flourishing online applications by suggesting appealing items on users’ next interactions, where global dependencies among items have proven to be indispensable for enhancing the quality of item representations toward a better understanding of user dynamic preferences. Existing methods rely on pre-defined graphs with shallow Graph Neural Networks to capture such necessary dependencies due to the constraint of the over-smoothing problem. However, this graph representation learning paradigm makes them difficult to satisfy the original expectation because of noisy graph structures and the limited ability of shallow architectures for modeling high-order relations. In this paper, we propose a novel Graph Diffusion Representation-enhanced Attention Network for sequential recommendation, which explores the construction of deeper networks by utilizing graph diffusion on adaptive graph structures for generating expressive item representations. Specifically, we design an adaptive graph generation strategy via leveraging similarity learning between item embeddings, automatically optimizing the input graph topology under the guidance of downstream recommendation tasks. Afterward, we propose a novel graph diffusion paradigm with robustness to over-smoothing, which enriches the learned item representations with sufficient global dependencies for attention-based sequential modeling. Moreover, extensive experiments demonstrate the effectiveness of our approach over state-of-the-art baselines.

Review-based recommender systems explore semantic aspects of users’ preferences by incorporating user-generated reviews into rating-based models. Recent works have demonstrated the potential of review information to improve the recommendation capacity. However, most existing studies rely on optimizing review-based representation learning part, thus failing to explicitly capture the fine-grained semantic aspects, and also ignoring the intrinsic correlation between ratings and reviews. To address these problems, we propose a multi-aspect graph contrastive learning framework, named MAGCL, with three distinctive designs: (i) a multi-aspect representation learning module, which projects semantic relations to different subspaces by decoupling review information, and then obtains high-order decoupled representations in each aspect via graph encoder. (ii) the contrastive learning module performs graph contrastive learning to capture the correlation between rating and review patterns, which utilize unlabeled data to generate self-supervised signals and, in turn, relieve the data sparsity problem of supervision signals. (iii) the multi-task learning module conducts joint training to learn high-order structure-aware yet self-discriminative node representations by combining recommendation task and self-supervised task, which helps alleviate the over-smoothing problem. Extensive experiments are conducted on four real-world review datasets and the results show the superiority of the proposed framework MAGCL compared with several state of the arts. We also provide further analysis on multi-aspect representations and graph contrastive learning to verify the advantage of proposed framework.

Cross-domain recommendation (CDR), which leverages information collected from other domains, has been empirically demonstrated to effectively alleviate data sparsity and cold-start problems encountered in traditional recommendation systems. However, current CDR methods, including those considering time information, do not jointly model the general and current interests within and across domains, which is pivotal for accurately predicting users’ future interactions. In this article, we propose a Contrastive learning-enhanced Multi-View interest learning model (CMVCDR) for cross-domain sequential recommendation. Specifically, we design a static view and a sequential view to model uses’ general interests and current interests, respectively. We divide a user’s general interest representation into a domain-invariant part and a domain-specific part. A cross-domain contrastive learning objective is introduced to impose constraints for optimizing these representations. In the sequential view, we first devise an attention mechanism guided by users’ domain-invariant interest representations to distill cross-domain knowledge pertaining to domain-invariant factors while reducing noise from irrelevant factors. We further design a domain-specific interest-guided temporal information aggregation mechanism to generate users’ current interest representations. Extensive experiments demonstrate the effectiveness of our proposed model compared with state-of-the-art methods.

User cold-start recommendation is one of the most challenging problems that limit the effectiveness of recommender systems. Meta-learning-based methods are introduced to address this problem by learning initialization parameters for cold-start tasks. Recent studies attempt to enhance the initialization methods. They first represent each task by the cold-start user and interacted items. Then they distinguish tasks based on the task relevance to learn adaptive initialization. However, this manner is based on the assumption that user preferences can be reflected by the interacted items saliently, which is not always true in reality. In addition, we argue that previous approaches suffer from their adaptive framework (e.g., adaptive initialization), which reduces the adaptability in the process of transferring meta-knowledge to personalized RSs. In response to the issues, we propose a task-difficulty-aware meta-learning with adaptive update strategies (TDAS) for user cold-start recommendation. First, we design a task difficulty encoder, which can represent user preference salience, task relevance, and other task characteristics by modeling task difficulty information. Second, we adopt a novel framework with task-adaptive local update strategies by optimizing the initialization parameters with task-adaptive per-step and per-layer hyperparameters. Extensive experiments based on three real-world datasets demonstrate that our TDAS outperforms the state-of-the-art methods. The source code is available at https://github.com/XuHao-bit/TDAS.

The issue of user cold-start poses a long-standing challenge to recommendation systems, due to the scarce interactions of new users. Recently, meta-learning based studies treat each cold-start user as a user-specific few-shot task and then derive meta-knowledge about fast model adaptation across training users. However, existing solutions mostly do not clearly distinguish the concept of new users and the concept of novel preferences, leading to over-reliance on meta-learning based adaptability to novel patterns. In addition, we also argue that the existing meta-training task construction inherently suffers from the memorization overfitting issue, which inevitably hinders meta-generalization to new users. In response to the aforementioned issues, we propose a preference learning decoupling framework, which is enhanced with meta-augmentation (PDMA), for user cold-start recommendation. To rescue the meta-learning from unnecessary adaptation to common patterns, our framework decouples preference learning for a cold-start user into two complementary aspects: common preference transfer, and novel preference adaptation. To handle the memorization overfitting issue, we further propose to augment meta-training users by injecting attribute-based noises, to achieve mutually-exclusive tasks. Extensive experiments on benchmark datasets demonstrate that our framework achieves superior performance improvements against state-of-the-art methods. We also show that our proposed framework is effective in alleviating memorization overfitting.

Next Point-of-Interest (POI) recommendation is an essential part of the flourishing location-based applications, where the demands of users are not only conditioned by their recent check-in behaviors but also by the critical influence stemming from geographical dependencies among POIs. Existing methods leverage Graph Neural Networks with the aid of pre-defined POI graphs to capture such indispensable correlations for modeling user preferences, assuming that the appropriate geographical dependencies among POIs could be pre-determined. However, the pre-defined graph structures are always far from the optimal graph topology due to noise and adaptability issues, which may decrease the expressivity of learned POI representations as well as the credibility of modeling user preferences. In this paper, we propose a novel Adaptive Graph Representation-enhanced Attention Network (AGRAN) for next POI recommendation, which explores the utilization of graph structure learning to replace the pre-defined static graphs for learning more expressive representations of POIs. In particular, we develop an adaptive POI graph matrix and learn it via similarity learning with POI embeddings, automatically capturing the underlying geographical dependencies for representation learning. Afterward, we incorporate the learned representations of POIs and personalized spatial-temporal information with an extension to the self-attention mechanism for capturing dynamic user preferences. Extensive experiments conducted on two real-world datasets validate the superior performance of our proposed method over state-of-the-art baselines.

Graph Convolutional Networks (GCNs) have been widely used for collaborative filtering, due to their effectiveness in exploiting high-order collaborative signals. However, two issues have not been well addressed by existing studies. First, usually only one kind of information is utilized, i.e., user preference in user-item graphs or item dependency in item-item graphs. Second, they usually adopt static graphs, which cannot retain the temporal evolution of the information. These can limit the recommendation quality. To address these limitations, we propose to mine three kinds of information (user preference, item dependency, and user behavior similarity) and their temporal evolution by constructing multiple discrete dynamic heterogeneous graphs (i.e., a user-item dynamic graph, an item-item dynamic graph, and a user-subseq dynamic graph) from interaction data. A novel network (PDGCN) is proposed to learn the representations of users and items in these dynamic graphs. Moreover, we designed a structural neighbor aggregation module with novel pooling and convolution operations to aggregate the features of structural neighbors. We also design a temporal neighbor aggregation module based on self-attention mechanism to aggregate the features of temporal neighbors. We conduct extensive experiments on four real-world datasets. The results indicate that our approach outperforms several competing methods in terms of Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG). Dynamic graphs are also shown to be effective in improving recommendation performance.

Textual reviews contain rich semantic information that is useful for making better recommendation, as such semantic information may indicate more fine-grained preferences of users. Recent efforts make considerable improvement on recommendation by integrating textual reviews in rating-based recommendations. However, there still exist major challenges on integrating textual reviews for recommendation. On the one hand, most existing works focus on learning a single representation from reviews but ignoring complex relations between users (or items) and reviews, which may fail to capture user preferences and item attributes together. On the other hand, these works independently learn latent representations from ratings and reviews while omitting correlations between rating-based features and review-based features, which may harm recommendation performance. In this article, we capture the aspect-aware relations by constructing heterogeneous graphs from reviews. Furthermore, we propose a new recommendation model, namely AHOR, to jointly distill rating-based features and review-based features, which are derived from ratings and reviews, respectively. To explore the multi-hop connectivity information between users, items, and aspects, a novel graph neural network is introduced to learn aspect-aware high-order representations. Experiments based on public datasets show that our approach outperforms state-of-the-art methods. We also provide detailed analysis on the high-order signals and the aspect importance to show the interpretability of our proposed model.

User cold-start scenarios pose great challenges to recommendation systems in accurately capturing user preferences with sparse interaction records. Besides incorporating auxiliary information to enrich user/item representations, recent studies under the schema of meta-learning focus on quickly adapting personalized recommendation models based on cold-start users’ scarce interactions. The majority of meta-learning based recommendation methods follow a bi-level optimization paradigm and learn globally shared initialization across all cold-start recommendation tasks. In addition, to further facilitate the ability of fast adaptation, existing methods have made efforts to tailor task-specific prior knowledge by identifying the individual characteristics of each task. However, we argue that multi-view commonalities between existing users and cold-start users are also essential for precisely distinguishing new tasks, but not comprehensively modeled in previous studies. In this article, we propose a multifaceted relation-aware meta-learning approach namely MeCM for user cold-start recommendation, which enhances task-adaptive initialization customization by extracting multiple views of task relevance. We design a dual customization framework consisting of two successive phases including cluster-level customization and task-level customization. Specifically, MeCM first extracts multifaceted semantic relations between tasks and refines task commonalities into task clusters maintained with memory networks (MNs). Globally learned fast weights corresponding to task clusters are queried to perform cluster-level customization. Then task-level customization is triggered based on contextual information of the target task via interaction-wise encoding. Extensive experiments on real-world datasets demonstrate the superior performance of our model over state-of-the-art meta-learning-based recommendation methods.

Next Point-of-Interest (POI) recommendation plays a critical role in many location-based applications as it provides personalized suggestions on attractive destinations for users. Since users' next movement is highly related to the historical visits, sequential methods such as recurrent neural networks are widely used in this task for modeling check-in behaviors. However, existing methods mainly focus on modeling the sequential regularity of check-in sequences but pay little attention to the intrinsic characteristics of POIs, neglecting the entanglement of the diverse influence stemming from different aspects of POIs. In this paper, we propose a novel Disentangled Representation-enhanced Attention Network (DRAN) for next POI recommendation, which leverages the disentangled representations to explicitly model different aspects and corresponding influence for representing a POI more precisely. Specifically, we first design a propagation rule to learn graph-based disentangled representations by refining two types of POI relation graphs, making full use of the distance-based and transition-based influence for representation learning. Then, we extend the attention architecture to aggregate personalized spatio-temporal information for modeling dynamic user preferences on the next timestamp, while maintaining the different components of disentangled representations independent. Extensive experiments on two real-world datasets demonstrate the superior performance of our model to state-of-the-art approaches. Further studies confirm the effectiveness of DRAN in representation disentanglement.

The task of next Point-of-Interest (POI) recommendation aims at recommending a list of POIs for a user to visit at the next timestamp based on his/her previous interactions, which is valuable for both location-based service providers and users. Recent state-of-the-art studies mainly employ recurrent neural network (RNN) based methods to model user check-in behaviors according to user’s historical check-in sequences. However, most of the existing RNN-based methods merely capture geographical influences depending on physical distance or successive relation among POIs. They are insufficient to capture the high-order complex geographical influences among POI networks, which are essential for estimating user preferences. To address this limitation, we propose a novel Graph-based Spatial Dependency modeling (GSD) module, which focuses on explicitly modeling complex geographical influences by leveraging graph embedding. GSD captures two types of geographical influences, i.e., distance-based and transition-based influences from designed POI semantic graphs. Additionally, we propose a novel Graph-enhanced Spatial-Temporal network (GSTN), which incorporates user spatial and temporal dependencies for next POI recommendation. Specifically, GSTN consists of a Long Short-Term Memory (LSTM) network for user-specific temporal dependencies modeling and GSD for user spatial dependencies learning. Finally, we evaluate the proposed model using three real-world datasets. Extensive experiments demonstrate the effectiveness of GSD in capturing various geographical influences and the improvement of GSTN over state-of-the-art methods.

Sequential recommendation which aims to predict a user's next interaction based on his/her previous behaviors, has attracted great attention. Recent studies mainly employ deep recurrent neural networks or self-attention networks to capture dynamic user preferences. However, existing methods merely focus on modeling users' clear interests in interacted items. We argue that for an interaction, the user may also have ambiguous interests in items that are semantically related to the interacted one. For comprehensively capturing user preferences, it is beneficial to discover potential interests from historical interactions at a broader itemset level. Therefore, in this paper, we propose a knowledge graph enhanced sequential recommendation model namely KGIE, which focuses on enhancing user interest modeling with knowledge-enriched itemsets by incorporating the knowledge graph. Specifically, in addition to item-level interest modeling with interacted items, we further construct knowledge-enriched itemsets that are extracted via high-order knowledge associations with the interacted items. For capturing personalized itemset-level interests, we design an attentive aggregation unit to combine item embeddings considering both inherent and contextual personalization signals. Furthermore, to balance the contributions of both two levels of interest modeling, we adaptively learn high-level preference representations with a gating fusion unit. Extensive experiments on three real-world datasets demonstrate the superior performance beyond state-of-the-art methods and recommendation interpretability of our model.

Air pollution is an important environmental issue of increasing concern, which impacts human health. Accurate air quality prediction is crucial for avoiding people suffering from serious air pollution. Most of the prior works focus on capturing the temporal trend of air quality for each monitoring station. Recent deep learning based methods also model spatial dependencies among neighboring stations. However, we observe that besides geospatially adjacent stations, the stations which share similar functionalities or consistent temporal patterns could also have strong dependencies. In this paper, we propose an Attentive Temporal Graph Convolutional Network (ATGCN) to model diverse inter-station relationships for air quality prediction of citywide stations. Specifically, we first encode three types of relationships among stations including spatial adjacency, functional similarity, and temporal pattern similarity into graphs. Then we design parallel encoding modules, which respectively incorporate attentive graph convolution operations into the Gated Recurrent Units (GRUs) to iteratively aggregate features from related stations with different graphs. Furthermore, augmented with an attention-based fusion unit, decoding modules with a similar structure to the encoding modules are designed to generate multi-step predictions for all stations. The experiments on two real-world datasets demonstrate the superior performance of our model beyond state-of-the-art methods.