Spectral graph neural networks (GNNs) are highly effective in modeling graph signals, with their success in recommendation often attributed to low-pass filtering. However, recent studies highlight the importance of high-frequency signals. The role of low-frequency and high-frequency graph signals in recommendation remains unclear. This paper aims to bridge this gap by investigating the influence of graph signals on recommendation performance. We theoretically prove that the effects of low-frequency and high-frequency graph signals are equivalent in recommendation tasks, as both contribute by smoothing the similarities between user-item pairs. To leverage this insight, we propose a frequency signal scaler, a plug-and-play module that adjusts the graph signal filter function to fine-tune the smoothness between user-item pairs, making it compatible with any GNN model. Additionally, we identify and prove that graph embedding-based methods cannot fully capture the characteristics of graph signals. To address this limitation, a space flip method is introduced to restore the expressive power of graph embeddings. Remarkably, we demonstrate that either low-frequency or high-frequency graph signals alone are sufficient for effective recommendations. Extensive experiments on four public datasets validate the effectiveness of our proposed methods. Code is avaliable at https://github.com/mojosey/SimGCF.

Trajectory anomaly detection, as a fundamental operation for moving object pattern discovery, plays an irreplaceable and critical role in spatio-temporal location-based services. Conducting online detection based on the current positions and their contextual semantics can significantly enhance the value of trajectory data. However, existing approaches suffer from two fundamental limitations: 1) treat trajectories as indivisible sequences or apply rigid segmentation strategies, and 2) use of a single detection model that struggles to adapt to concept drift caused by evolving trajectory distributions. Such limitations make it impossible to detect abnormal trajectories in a timely and semantically comprehensive manner. To fill this gap, we propose Stamp, a novel framework for Semantic-aware sub-Trajectory Anomaly detection with a diffusion Multi-model Pool. In particular, Stamp comprises three key innovations: 1. It employs a semantic-driven dynamic segmentation mechanism that identifies natural breakpoints in trajectories based on changes in road semantics, rather than fixed rules. 2. It enhances trajectory representation by embedding road network semantic vectors, capturing both spatial geometry and functional urban characteristics. 3. It employs a pool of diffusion models that dynamically evolves through reliability assessment, similarity measurement, and strategic merging operations, ensuring adaptability to concept drift while leveraging the superior generative capabilities of diffusion models over traditional autoencoders. Experimental results demonstrate that Stamp improves detection efficiency by 35%, AUPR by 5.6%, and F1-score by 2.7% on two large-scale real-world urban trajectory datasets when compared to state-of-the-art methods, demonstrating its effectiveness for real-time anomaly detection in complex urban environments.

Knowledge graphs (KGs) have been widely adopted to mitigate data sparsity and address cold-start issues in recommender systems. While existing KGs-based recommendation methods can predict user preferences and demands, they fall short in generating explicit recommendation paths and lack explainability. As a step beyond the above methods, recent advancements utilize reinforcement learning (RL) to find suitable items for a given user via explainable recommendation paths. However, the performance of these solutions is still limited by the following two points. (1) Lack of ability to capture contextual dependencies from neighboring information. (2) The excessive reliance on short recommendation paths due to efficiency concerns. To surmount these challenges, we propose a category-aware dual-agent reinforcement learning (CADRL) model for explainable recommendations over KGs. Specifically, our model comprises two components: (1) a category-aware gated graph neural network that jointly captures context-aware item representations from neighboring entities and categories, and (2) a dual-agent RL framework where two agents efficiently traverse long paths to search for suitable items. Finally, experimental results show that CADRL outperforms state-of-the-art models in terms of both effectiveness and efficiency on large-scale datasets.

Entity Alignment (EA) aims to identify equivalent entities across multiple Knowledge Graphs (KGs). However, when applied to larger-scale KGs, most existing EA approaches suffer from the scalability issue due to excessive GPU memory and time consumption. To mitigate this, recent advances have introduced the Large-scale EA (LsEA) task, which divides large-scale KG pairs into smaller sub-graph pairs. Despite their promising results, several notable challenges remain, preventing these advances from achieving optimal performance: 1) How to effectively utilize critical structures when generating sub-tasks? 2) How to supplement high-quality context to enhance LsEA performance? 3) How to address scenarios without alignment seeds? To tackle these challenges, we propose a novel method called ELsEA. It comprises three main components: (1) Source and Target Graph Partition, using a Metis-based weighted partitioner and a counter-part candidate generator to partition source and target graphs respectively, aiming to utilize critical structures effectively; (2) Supplement High-quality Context, which utilizes a value-based informativeness-evaluation module and a neighbor enrichment module to assess each entity's informativeness effectively, then supplement high-quality context based on this informativeness; and (3) Seed-free Setup, introducing a mixed-info pseudo-seed generation strategy to mitigate name bias, generating accurate pseudo-seeds when alignment seeds are unavailable. Extensive experiments demonstrate that ELsEA outperforms state-of-the-art baselines. The code of ELsEA is available online11https://githuh.com/wx-qzhou/ELsEA.git.

Group recommendation aims to recommend desired items for a group of users. Existing methods mainly adopt deterministic networks to represent groups as fixed-point vectors, assuming their preferences be highly close to these vectors in interest space. However, each group tends to have various interests, which cannot be fully captured by fixed-point vectors and thus calls for probabilistic modeling of interests as density instead. Although this can be supported by Variational AutoEncoder (VAE), interaction data in group recommendation are highly sparse and insufficient for VAE model training, resulting in high risks of posterior collapse and deficiency in personalization. To this end, this paper proposes a contrastive variational learning model boosted by variational model augmentation and an easyto-hard paradigm. Specifically, VAE with tailored attention is first employed to represent group preferences as variational vectors for probabilistic preference modeling. Additionally, we conduct data-agnostic augmentation via learnable variational dropout, which removes redundant or irrelevant neurons in VAE to generate meaningful augmented views adequately for contrastive learning in spite of data sparsity. Difficulty-aware negative sampling is further applied to generate high-quality negative samples adapting to varying requirements of task difficulty according to the training process. Finally, we utilize density-based variational alignment to guide the optimization process of contrastive learning. Experiments on four real-world datasets are conducted to demonstrate the significant performance improvements of our model compared with SOTA methods for group recommendation.

Recommender systems play a crucial role in delivering personalized services to users, but the increasing volume of user data raises significant concerns about privacy, security, and utility. However, existing machine unlearning methods cannot be directly applied to recommendation systems as they overlook the collaborative information shared across users and items. More recently, a method known as RecEraser was introduced, offering partitioning and aggregation-based approaches. Nevertheless, these approaches have limitations due to their inadequate handling of additional overhead costs. In this paper, we propose A General Strategy Graph Collaborative Filtering for Recommendation Unlearning (GSGCF-RU), which is a novel model-agnostic learnable delete operator that optimizes unlearning edge consistency and feature representation consistency. Specifically, the GSGCF-RU model utilizes unlearning edge consistency to eliminate the influence of deleted elements, followed by feature representation consistency to retain knowledge after deletion. Lastly, experimental results on three real-world public benchmarks demonstrate that GSGCF-RU not only achieves efficient recommendation unlearning but also surpasses state-of-the-art methods in terms of model utility. The source code can be found at https://github.com/YongjingHao/GSGCF-RU.

With the popularization of mobile applications and the timely acquisition of fresh data, real-time clustering and its evolution analysis have become the primary operations for data processing and knowledge discovery. Such continuous queries on massive objects are computation-intensive tasks in dynamic scenarios. However, existing clustering techniques are incompetent to achieve decent performance when computation-intensive operations frequently occur in streaming scenarios, which is caused by two challenges: (i) uncertainty of the clustering frequency; (ii) unpredictable distribution evolution. Hence, it is critical to find a lightweight model that can cluster the high-speed dynamic instances while exploiting the evolution amid different clustering results. This paper focuses on the problem of real-time clustering on streaming data in computation-intensive and high-dynamics tasks, through a framework Ocean, consisting of the Online clustering algorithm and evolution analysis. Particularly, the framework conceives a flexible composite window to augment the knowledge mining, achieving a proper real-time response in various scenarios. The evolution analysis supports full life-cycle detection, improving the adaptability to dynamic concept drifts and multiple patterns. Inspired by the grid partition strategy, this framework adopts grid feature vectors to capture the significant changes in streaming data. Furthermore, we propose an optimization that removes sparse grids timely and performs the online clustering adaptively for space and time efficiency. It is proven to be effective both theoretically and experimentally. This strategy enables real-time clustering for dynamic streaming data without degrading the clustering quality or increasing the computation cost. Experiments on real datasets and synthetic datasets verify the accuracy and effectiveness of Ocean compared to the state-of-the-art approaches, as well as the superior ability to perform clustering in a real-time manner.

Sequential Recommendation (SR) has received increasing attention due to its ability to capture user dynamic preferences. Recently, Contrastive Learning (CL) provides an effective approach for sequential recommendation by learning invariance from different views of an input. However, most existing data or model augmentation methods may destroy semantic sequential interaction characteristics and often rely on the hand-crafted property of their contrastive view-generation strategies. In this paper, we propose a Meta-optimized Seq2Seq Generator and Contrastive Learning (Meta-SGCL) for sequential recommendation, which applies the meta-optimized two-step training strategy to adaptive generate contrastive views. Specifically, Meta-SGCL first introduces a simple yet effective augmentation method called Sequence-to-Sequence (Seq2Seq) generator, which treats the Variational AutoEncoders (VAE) as the view generator and can constitute contrastive views while preserving the original sequence's semantics. Next, the model employs a meta-optimized two-step training strategy, which aims to adaptively generate contrastive views without relying on manually designed view-generation techniques. Finally, we evaluate our proposed method Meta-SGCL using three public real-world datasets. Compared with the state-of-the-art methods, our experimental results demonstrate the effectiveness of our model and the code is available.11https.//anonymous.4open.science/status/Meta-SGCL-05B5

Graph Collaborative Filtering (GCF) is designed to leverage high-order connectivity in user-item graphs, thereby significantly enhancing recommendation performance. Recent advancements have seen the integration of contrastive learning into GCF as a strategy to mitigate the challenges of data sparsity. This approach involves creating contrastive views through augmentations, followed by the generation of self-supervised signals. These signals are produced by maximizing the mutual information between the contrastive views. While this method has proven effective, we argue that current CL-based GCF models are still limited to current augmentation techniques. Existing data augmentation or noise perturbation may destroy the structural and semantic features of the original data and node attribute information is not considered. To tackle the above limitations, we propose a Meta-optimized Structure and Semantic Contrastive Learning for Graph Collaborative Filtering, named Meta-SSCL, which utilizes graph structure information and semantic information contrastive learning for recommendation. Specifically, we first model the structural and node semantic information representations with LightGCN and vanilla attention mechanism, respectively. Then consider the structural and semantic information as two contrastive views for recommendation. Next, the meta-optimized two-step training strategy generates adaptive contrastive views. Finally, we fuse structural and semantic representations for recommendation. Extensive experiments on real-world datasets demonstrate that Meta-SSCL consistently outperforms state-of-the-art sequential recommendation methods. The code is available1.

ID-based Recommender Systems (RecSys), where each item is assigned a unique identifier and subsequently converted into an embedding vector, have dominated the de-signing of RecSys. Though prevalent, such ID-based paradigm is not suitable for developing transferable RecSys and is also susceptible to the cold -start issue. In this paper, we unleash the boundaries of the ID- based paradigm and propose a Pure Multi-Modality based Recommender system (PMMRec), which relies solely on the multi-modal contents of the items (e.g., texts and images) and learns transition patterns general enough to transfer across domains and platforms. Specifically, we design a plug-and-play framework architecture consisting of multi-modal item encoders, a fusion module, and a user encoder. To align the cross-modal item representations, we propose a novel next-item enhanced cross-modal contrastive learning objective, which is equipped with both inter- and intra-modality negative samples and explicitly incorporates the transition patterns of user behaviors into the item encoders. To ensure the robustness of user representations, we propose a novel noised item detection objective and a robustness-aware contrastive learning objective, which work together to denoise user sequences in a self-supervised manner. PMMRec is designed to be loosely coupled, so after being pre-trained on the source data, each component can be transferred alone, or in conjunction with other components, allowing PMMRec to achieve versatility under both multi-modality and single-modality transfer learning settings. Extensive experiments on 4 sources and 10 target datasets demonstrate that PMMRec surpasses the state-of-the-art recommenders in both recommendation performance and transferability. Our code and dataset is available at: https://github.com/ICDE24IPMMRec.

Multi-modal data have recently been utilized to improve the performance of knowledge graph completion (KGC), attracting widespread research interest. However, they have been ignored in few-shot knowledge graph completion (FKGC), which aims to discover potential facts involving unseen relations that only appear in few-shot triples. The most relevant FKGC study simply concatenates various modal features, but the performance is still limited due to the following problems: (1) lack of exploiting significant multi-modal features in neighborhoods, and (2) ineffectively modeling inter-modal interactions in a few-shot setting. To tackle these problems, we propose a novel relational learning model entitled MMSN (Multi-Modal Siamese Network) for few-shot knowledge graph completion, which is composed of the following two primary modules: the Siamese multi-modal neighbor encoder (SMNE) and the meta-learning multi-modal knowledge representation decoder (MKRD). The module SMNE is developed to encode diverse modalities of neighbors by a Siamese attention network, fuse multi-modal information through a gating fusion network, and learn effective relational embeddings using an aggregator. The module MKRD is introduced to handle inter-modal interactions between multiple modalities and train the proposed model in a few-shot scenario. Extensive experiments demonstrate that our proposed model MMSN outperforms the state-of-the-art FKGC models, including uni-modal and multi-modal models, on two real-world few-shot multi-modal datasets.

Group recommendation aims to recommend desired items for a group of users. Due to the sparsity of group-item interactions, existing methods mainly model group preferences by aggregating member-level preference. However, they not only ignore possible user interest drift in specific groups, but also adopt deterministic models to represent group preferences using fixed-points, which are weak in characterizing uncertain group preferences. To this end, following the paradigm of variational learning, this paper proposes a multi-view attentive variational preference aggregation network called GroupAV for group rec-ommendation, so as to conduct user/group preference modeling and aggregation in a density-based manner. Specifically, we first adopt Variational AutoEncoder (VAE) to capture member-level preferences by variational vectors as density. To address user interest drift in groups, a variational preference adapter module is designed to learn group-contextualized preferences via rational transformation in variational space. Next, attentive variational aggregation networks are carefully designed for group-level preference aggregation in two different views (i.e., group-interactions and member-consensus views). Besides, we apply contrastive learning and gating fusion to optimize the multi-view learning process for the final group preference modeling of Mixture-of-Gaussian distribution. Finally, we conduct experiments on real-world datasets and demonstrate GroupAV's significant performance improvements compared to state-of-the-art group recommendation methods.

The user purchase behaviors are mainly influenced by their intentions (e.g., buying clothes for decoration, buying brushes for painting, etc.). Modeling a user's latent intention can significantly improve the performance of recommendations. Previous works model users' intentions by considering the predefined label in auxiliary information or introducing stochastic data augmentation to learn purposes in the latent space. However, the auxiliary information is sparse and not always available for recommender systems, and introducing stochastic data augmentation may introduce noise and thus change the intentions hidden in the sequence. Therefore, leveraging user intentions for sequential recommendation (SR) can be challenging because they are frequently varied and unobserved. In this paper, Intent contrastive learning with Cross Subsequences for sequential Recommendation (ICSRec) is proposed to model users' latent intentions. Specifically, ICSRec first segments a user's sequential behaviors into multiple subsequences by using a dynamic sliding operation and takes these subsequences into the encoder to generate the representations for the user's intentions. To tackle the problem of no explicit labels for purposes, ICSRec assumes different subsequences with the same target item may represent the same intention and proposes a coarse-grain intent contrastive learning to push these subsequences closer. Then, fine-grain intent contrastive learning is mentioned to capture the fine-grain intentions of subsequences in sequential behaviors. Extensive experiments conducted on four real-world datasets demonstrate the superior performance of the proposed ICSRec model compared with baseline methods.

Graph collaborative filtering (GCF) has achieved exciting recommendation performance with its ability to aggregate high-order graph structure information. Recently, contrastive learning (CL) has been incorporated into GCF to alleviate data sparsity and noise issues. However, most of the existing methods employ random or manual augmentation to produce contrastive views that may destroy the original topology and amplify the noisy effects. We argue that such augmentation is insufficient to produce the optimal contrastive view, leading to suboptimal recommendation results. In this article, we proposed a L earnable M odel A ugmentation C ontrastive L earning (LMACL) framework for recommendation, which effectively combines graph-level and node-level collaborative relations to enhance the expressiveness of collaborative filtering (CF) paradigm. Specifically, we first use the graph convolution network (GCN) as a backbone encoder to incorporate multi-hop neighbors into graph-level original node representations by leveraging the high-order connectivity in user-item interaction graphs. At the same time, we treat the multi-head graph attention network (GAT) as an augmentation view generator to adaptively generate high-quality node-level augmented views. Finally, joint learning endows the end-to-end training fashion. In this case, the mutual supervision and collaborative cooperation of GCN and GAT achieves learnable model augmentation. Extensive experiments on several benchmark datasets demonstrate that LMACL provides a significant improvement over the strongest baseline in terms of Recall and NDCG by 2.5%–3.8% and 1.6%–4.0%, respectively. Our model implementation code is available at https://github.com/LiuHsinx/LMACL .

Sequential recommendation aims to predict the next item of interest to users based on their historical behavior data. Usually, users’ global and local preferences jointly affect the final recommendation result in different ways. Most existing works use transformers to globally model sequences, which makes them face the dilemma of quadratic computational complexity when dealing with long sequences. Moreover, the scope setting of the user’s local preference is usually static and single, and cannot cover richer multi-level local semantics. To this end, we proposed a parallel architecture for capturing global representation and M ulti-granularity L ocal dependencies with M LP for sequential Rec ommendation ( MLM4Rec ). For global representation, we utilize modified MLP-Mixer to capture global information of user sequences due to its simplicity and efficiency. For local representation, we incorporate convolution into MLP and propose a multi-granularity local awareness mechanism for capturing richer local semantic information. Moreover, we introduced a weight pooling method to adaptively fuse local-global representations instead of directly concatenation. Our model has the advantages of low complexity and high efficiency thanks to its simple MLP structure. Experimental results on three public datasets demonstrate the effectiveness of our proposed model. Our code is available here 1 .

Contrastive learning with Transformer-based sequence encoder has gained predominance for sequential recommendation due to its ability to mitigate the data noise and the data sparsity issue. However, existing contrastive learning approaches for sequential recommendation still suffer from two limitations. First, they mainly center on left-to-right unidirectional Transformers as base encoders, which are suboptimal for sequential recommendation because user behaviors may not be a rigid left-to-right sequence. Second, they devise contrastive learning objectives only from the sequence level, neglecting the rich self-supervision signals from the feature level. To address these limitations, we propose a novel framework called Feature-aware Contrastive Learning with bidirectional Transformers for sequential Recommendation (FCLRec) to effectively leverage feature information for sequential recommendation. Specifically, we first augment bidirectional Transformers with a novel feature-aware self-attention module that is able to simultaneously model the complex relationships between sequences and features. Next, we propose a novel feature-aware contrastive learning objective that generates a collection of positive samples via three types of augmentations from three different levels. Finally, we adopt feature prediction as an auxiliary task to strengthen the connections between items and features. Our experimental results on four public benchmark datasets show that FCLRec outperforms the state-of-the-art methods for sequential recommendation.

Sequential Recommendation (SR) methods play a crucial role in recommender systems, which aims to capture users' dynamic interest from their historical interactions. Recently, Contrastive Learning (CL), which has emerged as a successful method for sequential recommendation, utilizes various data augmentations to generate contrastive views to mine supervised signals from data to alleviate data sparsity issues. However, most existing sequential data augmentation methods may destroy semantic sequential interaction characteristics. Meanwhile, they often adopt random operations when generating contrastive views leading to suboptimal performance. To this end, in this paper, we propose a Learnable Model Augmentation Contrastive learning for sequential Recommendation (LMA4Rec). Specifically, LMA4Rec first takes the model-based augmentation method to generate constructive views. Then, LMA4Rec uses Learnable Bernoulli Dropout (LBD) to implement learnable model augmentation operations. Next, contrastive learning is used between the contrastive views to extract supervised signals. Furthermore, a novel multi-positive contrastive learning loss alleviates the supervised sparsity issue. Finally, experiments on public datasets show that our LMA4Rec method effectively improved sequential recommendation performance compared with the state-of-the-art baseline methods.

Reinforcement learning (RL)-based multi-hop reasoning has become an interpretable way for knowledge graph reasoning owing to its persuasive explanations for the predicted results, but the reasoning performance of these methods drops significantly over few-shot relations (only contain few triplets). To address this problem, recent studies introduce meta-learning into RL-based reasoning methods. However, the performance of these studies is still limited due to the following points: (1) the overall reasoning accuracy is impaired due to the low reasoning accuracies over some hard relations; (2) the reasoning process becomes laborious and ineffective owing to the existence of noisy data; (3) the generalizability is negatively affected due to the lack of knowledge-sharing. To tackle these challenges, we propose a novel modelHMLSconsisting of two modulesHHML(HierarchicalHardness-awareMeta-reinforcementLearning) andHHS(HierarchicalHardness-awareSampling). Specifically,HHMLcontains the following two components: (1) a hardness-aware RL conducts multi-hop reasoning by training hardness-aware batches and reducing noise; (2) a knowledge-sharing meta-learning adapts to few-shot relations by exploiting common features in the hierarchical relation structure. The other moduleHHSgenerates hardness-aware batches from relation and relation-cluster levels. The experimental results demonstrate that this work notably outperforms the state-of-the-art approaches in few-shot scenarios.

Sequential recommendation (SR) aims to model user preferences by capturing behavior patterns from their item historical interaction data. Most existing methods model user preference in the time domain, omitting the fact that users’ behaviors are also influenced by various frequency patterns that are difficult to separate in the entangled chronological items. However, few attempts have been made to train SR in the frequency domain, and it is still unclear how to use the frequency components to learn an appropriate representation for the user. To solve this problem, we shift the viewpoint to the frequency domain and propose a novel Contrastive Enhanced SLIde Filter MixEr for Sequential Recommendation, named SLIME4Rec. Specifically, we design a frequency ramp structure to allow the learnable filter slide on the frequency spectrums across different layers to capture different frequency patterns. Moreover, a Dynamic Frequency Selection (DFS) and a Static Frequency Split (SFS) module are proposed to replace the self-attention module for effectively extracting frequency information in two ways. DFS is used to select helpful frequency components dynamically, and SFS is combined with the dynamic frequency selection module to provide a more fine-grained frequency division. Finally, contrastive learning is utilized to improve the quality of user embedding learned from the frequency domain. Extensive experiments conducted on five widely used benchmark datasets demonstrate our proposed model performs significantly better than the state-of-the-art approaches. Our code is available at https://github.com/sudaada/SLIME4Rec.

Inductive link prediction (ILP) is to predict links for unseen entities in emerging knowledge graphs (KGs), considering the evolving nature of KGs. A more challenging scenario is that emerging KGs consist of only unseen entities without any edge connected to original KGs, called as disconnected emerging KGs (DEKGs). Existing studies for DEKGs only focus on predicting enclosing links, i.e., predicting links inside the emerging KG. The bridging links, which carry the evolutionary information from the original KG to DEKG, have not been investigated by previous work so far. To fill in the gap, we propose a novel model entitled DEKG-ILP (Disconnected Emerging Knowledge Graph Oriented Inductive Link Prediction) that consists of the following two components. (1) The module CLRM (Contrastive Learning-based Relation-specific Feature Modeling) is developed to extract global relation-based semantic features that are shared between original KGs and DEKGs with a novel sampling strategy. (2) The module GSM (GNN-based Subgraph Modeling) is proposed to extract the local subgraph topological information around each link in KGs. The extensive experiments conducted on several benchmark datasets demonstrate that DEKG-ILP has obvious performance improvements compared with state-of-the-art methods for both enclosing and bridging link prediction.

The self-attention mechanism, which equips with a strong capability of modeling long-range dependencies, is one of the extensively used techniques in the sequential recommendation field. However, many recent studies represent that current self-attention based models are low-pass filters and are inadequate to capture high-frequency information. Furthermore, since the items in the user behaviors are intertwined with each other, these models are incomplete to distinguish the inherent periodicity obscured in the time domain. In this work, we shift the perspective to the frequency domain, and propose a novel Frequency Enhanced Hybrid Attention Network for Sequential Recommendation, namely FEARec. In this model, we firstly improve the original time domain self-attention in the frequency domain with a ramp structure to make both low-frequency and high-frequency information could be explicitly learned in our approach. Moreover, we additionally design a similar attention mechanism via auto-correlation in the frequency domain to capture the periodic characteristics and fuse the time and frequency level attention in a union model. Finally, both contrastive learning and frequency regularization are utilized to ensure that multiple views are aligned in both the time domain and frequency domain. Extensive experiments conducted on four widely used benchmark datasets demonstrate that the proposed model performs significantly better than the state-of-the-art approaches.

Sequential recommendation aims to capture users' dynamic interest and predicts the next item of users' preference. Most sequential recommendation methods use a deep neural network as sequence encoder to generate user and item representations. Existing works mainly center upon designing a stronger sequence encoder. However, few attempts have been made with training an ensemble of networks as sequence encoders, which is more powerful than a single network because an ensemble of parallel networks can yield diverse prediction results and hence better accuracy. In this paper, we present Ensemble Modeling with contrastive Knowledge Distillation for sequential recommendation (EMKD). Our framework adopts multiple parallel networks as an ensemble of sequence encoders and recommends items based on the output distributions of all these networks. To facilitate knowledge transfer between parallel networks, we propose a novel contrastive knowledge distillation approach, which performs knowledge transfer from the representation level via Intra-network Contrastive Learning (ICL) and Cross-network Contrastive Learning (CCL), as well as Knowledge Distillation (KD) from the logits level via minimizing the Kullback-Leibler divergence between the output distributions of the teacher network and the student network. To leverage contextual information, we train the primary masked item prediction task alongside the auxiliary attribute prediction task as a multi-task learning scheme. Extensive experiments on public benchmark datasets show that EMKD achieves a significant improvement compared with the state-of-the-art methods. Besides, we demonstrate that our ensemble method is a generalized approach that can also improve the performance of other sequential recommenders. Our code is available at this link: https://github.com/hw-du/EMKD.

Contrastive Learning (CL) performances as a rising approach to address the challenge of sparse and noisy recommendation data. Although having achieved promising results, most existing CL methods only perform either hand-crafted data or model augmentation for generating contrastive pairs to find a proper augmentation operation for different datasets, which makes the model hard to generalize. Additionally, since insufficient input data may lead the encoder to learn collapsed embeddings, these CL methods expect a relatively large number of training data (e.g., large batch size or memory bank) to contrast. However, not all contrastive pairs are always informative and discriminative enough for the training processing. Therefore, a more general CL-based recommendation model called Meta-optimized Contrastive Learning for sequential Recommendation (MCLRec) is proposed in this work. By applying both data augmentation and learnable model augmentation operations, this work innovates the standard CL framework by contrasting data and model augmented views for adaptively capturing the informative features hidden in stochastic data augmentation. Moreover, MCLRec utilizes a meta-learning manner to guide the updating of the model augmenters, which helps to improve the quality of contrastive pairs without enlarging the amount of input data. Finally, a contrastive regularization term is considered to encourage the augmentation model to generate more informative augmented views and avoid too similar contrastive pairs within the meta updating. The experimental results on commonly used datasets validate the effectiveness of MCLRec.

Sequential recommendation has been a widely popular topic of recommender systems. Existing works have contributed to enhancing the prediction ability of sequential recommendation systems based on various methods, such as recurrent networks and self-attention mechanisms. However, they fail to discover and distinguish various relationships between items, which could be underlying factors which motivate user behaviors. In this article, we propose an Edge-Enhanced Global Disentangled Graph Neural Network (EGD-GNN) model to capture the relation information between items for global item representation and local user intention learning. At the global level, we build a global-link graph over all sequences to model item relationships. Then a channel-aware disentangled learning layer is designed to decompose edge information into different channels, which can be aggregated to represent the target item from its neighbors. At the local level, we apply a variational auto-encoder framework to learn user intention over the current sequence. We evaluate our proposed method on three real-world datasets. Experimental results show that our model can get a crucial improvement over state-of-the-art baselines and is able to distinguish item features.

Sequential recommendation, which aims to recommend next item that the user will likely interact in a near future, has become essential in various Internet applications. Existing methods usually consider the transition patterns between items, but ignore the transition patterns between features of items. We argue that only the item-level sequences cannot reveal the full sequential patterns, while explicit and implicit feature-level sequences can help extract the full sequential patterns. Meanwhile, the item-level sequential recommendation also suffers from limited supervised signal issues. In this article, we propose a novel model Feature-level Deeper Self-Attention Network with Contrastive Learning (FDSA-CL) for sequential recommendation. Specifically, FDSA-CL first integrates various heterogeneous features of items into feature-level sequences with different weights through a vanilla attention mechanism. After that, FDSA-CL applies separated self-attention blocks on item-level sequences and feature-level sequences, respectively, to model item transition patterns and feature transition patterns. Moreover, we propose contrastive learning and item feature recommendation tasks to capture the embedding commonality and further utilize the beneficial interaction among the two levels, so as to alleviate the sparsity of the supervised signal and extract the most critical information. Finally, we jointly optimize the above tasks. We evaluate the proposed model using two real-world datasets and experimental results show that our model significantly outperforms the state-of-the-art approaches.

Fastest route recommendation (FRR) is crucial for intelligent transportation systems. The existing methods treat it as a pathfinding problem on dynamic graphs, and extend A* algorithm with neuralized travel time estimators as cost functions. However, they fail to provide effective heuristic cost due to the neglect of its admissibility and the utilization of noise path information, resulting in sub-optimal results and inefficiency. Besides, path sequentiality is also ignored, affecting algorithm accuracy as well. In this paper, we propose a variational inference based fastest route recommendation method, which follows the framework of A* algorithm and provides effective costs for routing. Specifically, we first adopt a sequential estimator to accurately estimate the travel time of a specific path. More importantly, we design a variational inference based estimator, which models the distribution of travel time between two nodes and provides an effective heuristic cost with high probability of being admissible. We further take advantage of adversarial learning to enrich the fastest path information. To the best of our knowledge, we are the first to use variational estimator to consider the admissibility of heuristics in FRR. Extensive experiments are conducted on two real-world datasets. The results verify the performance advantage of our proposed method.

Contrastive learning with Transformer-based sequence encoder has gained predominance for sequential recommendation. It maximizes the agreements between paired sequence augmentations that share similar semantics. However, existing contrastive learning approaches in sequential recommendation mainly center upon left-to-right unidirectional Transformers as base encoders, which are suboptimal for sequential recommendation because user behaviors may not be a rigid left-to-right sequence. To tackle that, we propose a novel framework named Contrastive learning with Bidirectional Transformers for sequential recommendation (CBiT). Specifically, we first apply the slide window technique for long user sequences in bidirectional Transformers, which allows for a more fine-grained division of user sequences. Then we combine the cloze task mask and the dropout mask to generate high-quality positive samples and perform multi-pair contrastive learning, which demonstrates better performance and adaptability compared with the normal one-pair contrastive learning. Moreover, we introduce a novel dynamic loss reweighting strategy to balance between the cloze task loss and the contrastive loss. Experiment results on three public benchmark datasets show that our model outperforms state-of-the-art models for sequential recommendation. Our code is available at this link: https://github.com/hw-du/CBiT/tree/master.

With the prevalence of location-based services (LBS), trajectories are being generated rapidly. As is widely used in LBS, top-k trajectory similarity query serves as a key operation, deeply empowering applications such as travel route recommendation and carpooling. Given the rise of deep learning, trajectory representation has been well-proven to speed up this operator. However, existing representation-based computing modes remain two major problems understudied: the low quality of trajectory representation and insufficient support for various trajectory similarity metrics, which make them difficult to apply in practice. Therefore, we propose an Accurate metric-based representation learning approach for fast top-k trajectory similarity query, named Aries. Specifically, Aries has two sophisticated modules: (1) An novel trajectory embedding strategy enhanced by the bidirectional LSTM encoder and spatial attention mechanism, which can extract more precise and comprehensive knowledge. (2) A deep metric learning network aggregating multiple measures for better top-k query. Extensive experiments conducted on real trajectory dataset show that Aries achieves both impressive accuracy and lower training time compared with state-of-the-art solutions. In particular, it achieves 5x-10x speedup and 10%-20% accuracy improvement over Euclidean, Hausdorff, DTW, and EDR measures. Besides, our method can maintain stable performance when handling various scenarios, without repeated training in order to adapt to diverse similarity metrics.

Next Point-of-Interest (POI) recommendation which is of great value to both users and POI holders is a challenging task since complex sequential patterns and rich contexts are contained in extremely sparse user check-in data. Recently proposed embedding techniques have shown promising results in alleviating the data sparsity issue by modeling context information, and Recurrent Neural Network (RNN) has been proved effective in the sequential prediction. However, existing next POI recommendation approaches train the embedding and network model separately, which cannot fully leverage rich contexts. In this paper, we propose a novel unified neural network framework, named NeuNext, which leverages POI context prediction to assist next POI recommendation by joint learning. Specifically, the Spatio-Temporal Gated Network (STGN) is proposed to model personalized sequential patterns for users’ long and short term preferences in the next POI recommendation. In the POI context prediction, rich contexts on POI sides are used to construct graph, and enforce the smoothness among neighboring POIs. Finally, we jointly train the POI context prediction and the next POI recommendation to fully leverage labeled and unlabeled data. Extensive experiments on real-world datasets show that our method outperforms other approaches for next POI recommendation in terms of Accuracy and MAP.

Knowledge graph (KG) reasoning is a significant method for KG completion. To enhance the explainability of KG reasoning, some studies adopt reinforcement learning (RL) to complete the multi-hop reasoning. However, RL-based reasoning methods are severely limited by few-shot relations (only contain few triplets). To tackle the problem, recent studies introduce meta-learning into RL-based methods to improve reasoning performance. However, the generalization abilities of their models are limited due to the problem of low reasoning accuracies over hard relations (e.g., language and title). To overcome this problem, we propose a novel model called THML (Two-level Hardness-aware Meta-reinforcement Learning). Specifically, the model contains the following two components: (1) A hardness-aware meta-reinforcement learning method is proposed to predict the missing element by training hardness-aware batches. (2) A two-level hardness-aware sampling is proposed to effectively generate new hardness-aware batches from relation level and relation-cluster level. The generalization ability of our model is significantly improved by repeating the process of these two components in an alternate way. The experimental results demonstrate that THML notably outperforms the state-of-the-art approaches in few-shot scenarios.

Sequential recommendation has become an attractive topic in recommender systems. Existing sequential recommendation methods, including the methods based on the state-of-the-art self-attention mechanism, usually employ deterministic neural networks to represent user preferences as fixed-points in the latent feature spaces. However, the fixed-point vector lacks the ability to capture the uncertainty and dynamics of user preferences that are prevalent in recommender systems. In this paper, we propose a new Variational Self-Attention Network (VSAN), which introduces a variational autoencoder (VAE) into the self-attention network to capture latent user preferences. Specifically, we represent the obtained self-attention vector as density via variational inference, whose variance well characterizes the uncertainty of user preferences. Furthermore, we employ self-attention networks to learn the inference process and generative process of VAE, which well captures long-range and local dependencies. Finally, we evaluate our proposed method VSAN with two public real-world datasets. Our experimental results show the effectiveness of our model compared to the state-of-the-art approaches.

Destination prediction is known as an important problem for many location based services (LBSs). Existing solutions generally apply probabilistic models or neural network models to predict destinations over a subtrajectory, and adopt the standard attention mechanism to improve the prediction accuracy. However, the standard attention mechanism uses fixed feature representations, and has a limited ability to represent distinct features of locations. Besides, existing methods rarely take the impact of spatial and temporal characteristics of the trajectory into account. Their accuracies in fine-granularity prediction are always not satisfactory due to the data sparsity problem. Thus, in this paper, a carefully designed deep learning model called LATL model is presented. It not only adopts an adaptive attention network to model the distinct features of locations, but also implements time gates and distance gates into the Long Short-Term Memory (LSTM) network to capture the spatial-temporal relation between consecutive locations. Furthermore, to better understand the mobility patterns in different spatial granularities, and explore the fusion of multi-granularity learning capability, a hierarchical model that utilizes tailored combination of different neural networks under multiple spatial granularities is further proposed. Extensive empirical studies verify that the newly proposed models perform effectively and settle the problem nicely.

Recently accumulated massive amounts of geo-tagged photos provide an excellent opportunity to understand human behaviors and can be used for personalized tour recommendation. However, no existing work has considered the visual content information in these photos for tour recommendation. We believe the visual features of photos provide valuable information on measuring user / Point-of-Interest (POI) similarities, which is challenging due to data sparsity. To this end, in this paper, we propose a visual feature enhanced tour recommender system, named ‘Photo2Trip’, to utilize the visual contents and collaborative filtering models for recommendation. Specifically, we propose a Visual-enhanced Probabilistic Matrix Factorization model (VPMF), which integrates visual features into the collaborative filtering model, to learn user interests by leveraging the historical travel records. We then extend VPMF to End-to-End training framework to incorporate users (POIs) latent factors into the learning process of the visual content of photos, which generalizes the applicability of the proposed VPMF framework in tour recommendation. Extensive empirical studies verify that our proposed visual-enhanced personalized tour recommendation method outperforms other benchmark methods in terms of recommendation accuracy. The results also show that visual features are effective in alleviating the data sparsity and cold start problems on personalized tour recommendation.

Visual aesthetics of products plays an important role in the decision process when purchasing appearance-first products, e.g., clothes. Indeed, user’s aesthetic preference, which serves as a personality trait and a basic requirement, is domain independent and could be used as a bridge between domains for knowledge transfer. However, existing work has rarely considered the aesthetic information in product images for cross-domain recommendation. To this end, in this paper, we propose a new deep Aesthetic Cross-Domain Networks (ACDN), in which parameters characterizing personal aesthetic preferences are shared across networks to transfer knowledge between domains. Specifically, we first leverage an aesthetic network to extract aesthetic features. Then, we integrate these features into a cross-domain network to transfer users’ domain independent aesthetic preferences. Moreover, network cross-connections are introduced to enable dual knowledge transfer across domains. Finally, the experimental results on real-world datasets show that our proposed model ACDN outperforms benchmark methods in terms of recommendation accuracy.

An Attributed Dynamic Graph (ADG) contains multiple dynamic attributes associated with each edge. In ADG based applications, people usually can specify multiple constrains in the attributes to illustrate their requirements, such as the total cost, the total travel time and the stopover interval of a flight between two cities. This inspires a type of Multi-Constrained Temporal Path (MCTP) discovery in ADGs, which is a challenging NP-Complete problem. In order to deliver an efficient and effective temporal path discovery method to be used in real-time environment, we propose a Reinforcement Learning (RL) based, Monte Carlo Tree Search algorithm (RLMCTS). RL-MCTS uses a newly designed memory structure to address the challenges of Monte Carlo Tree Search (MCTS) in MCTP discovery. To the best of our knowledge, RL-MCTS is the first RL algorithm that supports path discovery in ADGs. The experimental results on ten real dynamic graphs demonstrate that our algorithm outperforms the state-of-the-art methods in terms of both efficiency and effectiveness.

The time signals behind a user's historical behaviors are important for better inferring what she prefers to interact with at the next time. For the attention-based recommendation methods, relative position encoding and time intervals division are two common ways to model the time signal behind each behavior. They either only consider the relative position of each behavior in the behavior sequence, or process the continuous temporal features into discrete category features for subsequent tasks, which can hardly capture the dynamic preferences of a user. In addition, although the existing recommendation methods have considered both long-term preference and short-term preference, they ignore the fact that the long-term preference of a user may be multi-faceted, and it is difficult to learn a user's fine-grained short-term preference. In this paper, we propose a Dynamic Multi-faceted Fine-grained Preference model (DMFP), where the multi-hops attention mechanism and the feature-level attention mechanism together with a vertical convolution operation are adopted to capture users' multi-faceted long-term preference and fine-grained short-term preference, respectively. Therefore, DMFP can better support the next item recommendation. Extensive experiments on three real-world datasets illustrate that our model can improve the effectiveness of the recommendation compared with the state-of-the-art methods.

The sequential recommendation, which models sequential behavioral patterns among users for the recommendation, plays a critical role in recommender systems. However, the state-of-the-art Recurrent Neural Networks (RNN) solutions rarely consider the non-linear feature interactions and non-monotone short-term sequential patterns, which are essential for user behavior modeling in sparse sequence data. In this paper, we propose a novel Recurrent Convolutional Neural Network model (RCNN). It not only utilizes the recurrent architecture of RNN to capture complex long-term dependencies, but also leverages the convolutional operation of Convolutional Neural Network (CNN) model to extract short-term sequential patterns among recurrent hidden states. Specifically, we first generate a hidden state at each time step with the recurrent layer. Then the recent hidden states are regarded as an “image”, and RCNN searches non-linear feature interactions and non-monotone local patterns via intra-step horizontal and inter-step vertical convolutional filters, respectively. Moreover, the output of convolutional filters and the hidden state are concatenated and fed into a fully-connected layer to generate the recommendation. Finally, we evaluate the proposed model using four real-world datasets from various application scenarios. The experimental results show that our model RCNN significantly outperforms the state-of-the-art approaches on sequential recommendation.

Destination prediction is known as an important problem for many location based services (LBSs). Existing solutions generally apply probabilistic models to predict destinations over a sub-trajectory, but their accuracies in fine-granularity prediction are always not satisfactory due to the data sparsity problem. This paper presents a carefully designed deep learning model called TALL model for destination prediction. It not only takes advantage of the bidirectional Long Short-Term Memory (LSTM) network for sequence modeling, but also gives more attention to meaningful locations that have strong correlations w.r.t. destination by adopting attention mechanism. Furthermore, a hierarchical model that explores the fusion of multi-granularity learning capability is further proposed to improve the accuracy of prediction. Extensive experiments on Beijing and Chengdu real datasets finally demonstrate that our proposed models outperform existing methods without considering external features.

With the rapid development of location-based social networks, Point-of-Interest (POI) recommendation has played an important role in helping people discover attractive locations. However, existing POI recommendation methods assume a flat structure of POIs, which are better described in a hierarchical structure in reality. Furthermore, we discover that both users' content and spatial preferences exhibit hierarchical structures. To this end, in this paper, we propose a hierarchical geographical matrix factorization model (HGMF) to utilize the hierarchical structures of both users and POIs for POI recommendation. Specifically, we first describe the POI influence degrees over regions with two-dimensional normal distribution, and learn the influence areas of different layers of POIs as the input of HGMF. Then, we perform matrix factorization on user content preference matrix, user spatial preference matrix, and POIs characteristic matrix jointly with the modeling of implicit hierarchical structures. Moreover, a two-step optimization method is proposed to learn the implicit hierarchical structure and find the solution of HGMF efficiently. Finally, we evaluate HGMF on two large-scale real-world location-based social networks datasets. Our experimental results demonstrate that it outperforms the state-of-the-art methods in terms of precision and recall.

Learning from imbalanced multilabel data is a challenging task. It has attracted considerable attention recently. In this paper we propose a MultiLabel Best First Over-sampling (ML-BFO) to improve the performance of multilabel classification algorithms, based on imbalance minimization and Wilson's ENN rule. Our experimental results show that ML-BFO not only duplicates fewer samples but also reduces the imbalance level much more than two state-of-the-art multilabel sampling methods, i.e., an over-sampling method LP-ROS and an under-sampling method MLeNN. Besides, ML-BFO significantly improves the performance of multilabel classification algorithms, and performs much better than LP-ROS and MLeNN.

Active learning is to select the most informative examples to request their labels. Most previous studies in active learning for multi-label classification didn't pay enough attention on label correlations. This leads to a bad performance for classification. In this paper, we proposed a chi-square statistics multi-label active learning (CSMAL) algorithm, which uses chi-square statistics to accurately evaluate correlations between labels. CSMAL considers not only positive relationships but also negative ones. It uses the average correlation between a potential label and its rest unlabeled labels as the label information for each sample-label pair. CSMAL further integrates uncertainty and label information to select example-label pairs to request labels. Our empirical results demonstrate that our proposed method CSMAL outperforms the state-of-the-art active learning methods for multi-label classification. It significantly reduces the labeling workloads and improves the performance of a classifier built.