Logical reasoning-based recommendation methods formulate logical expressions to characterize user-item interaction patterns, incorporating regularization constraints to ensure consistency with logical rules. However, these methods face two critical challenges: (1) As sequence length increases, they cannot effectively capture the dynamic transfer of user interests across subsequences (i.e., subsequence interest drift), thereby degenerating logical expressions to single-subsequence inference. (2) The time complexity of logical reasoning and rule learning scales quadratically with the sequence length, severely constraining computational efficiency in long-sequence recommendation. To address these challenges, we propose ELECTOR, an intErest-shift-aware long-sequence Logical reasoning for EffiCienT lOng-sequence Recommendation method. Specifically, we design a Subsequence Interest Learning Module (SIL) to model cross-subsequence interest drifts in long sequences. SIL employs a local attention mechanism to extract subsequence interests effectively and a global attention mechanism to capture the correlations among subsequence interests. Subsequently, we propose an Interest-aware Logical Reasoning (ILR) mechanism that performs logical reasoning using a limited set of subsequence and short-term interests, rather than reasoning over the entire sequence, significantly reducing time complexity. Additionally, ILR employs interest logical reasoning contrastive loss to ensure the model simultaneously considers multiple interests. Experiments on four real-world datasets demonstrate that our method significantly outperforms all baselines regarding computational efficiency and recommendation accuracy, confirming its effectiveness.

Multi-subject video generation aims to synthesize videos from textual prompts and multiple reference images, ensuring that each subject preserves natural scale and visual fidelity. However, current methods face two challenges: scale inconsistency, where variations in subject size lead to unnatural generation, and permutation sensitivity, where the order of reference inputs causes subject distortion. In this paper, we propose MoFu, a unified framework that tackles both challenges. For scale inconsistency, we introduce Scale-Aware Modulation (SMO), an LLM-guided module that extracts implicit scale cues from the prompt and modulates features to ensure consistent subject sizes. To address permutation sensitivity, we present a simple yet effective Fourier Fusion strategy that processes the frequency information of reference features via the Fast Fourier Transform to produce a unified representation. Besides, we design a Scale-Permutation Stability Loss to jointly encourage scale-consistent and permutation-invariant generation. To further evaluate these challenges, we establish a dedicated benchmark with controlled variations in subject scale and reference permutation. Extensive experiments demonstrate that MoFu significantly outperforms existing methods in preserving natural scale, subject fidelity, and overall visual quality.

Personalized image generation is crucial for improving the user experience, as it renders reference images into preferred ones according to user visual preferences. Although effective, existing methods face two main issues. First, existing methods treat all items in the user's historical sequence equally when extracting user preferences, overlooking the varying semantic similarities between historical items and the reference item. Disproportionately high weights for low-similarity items distort user visual preferences for the reference item. Second, existing methods heavily rely on consistency between generated and reference images to optimize generation, which leads to underfitting user preferences and hinders personalization. To address these issues, we propose Retrieval Augmented Personalized Image GenerAtion guided by Recommendation (RAGAR). Our approach uses a retrieval mechanism to assign different weights to historical items according to their similarities to the reference item, thereby extracting more refined users' visual preferences for the reference item. Then we introduce a novel rank task based on the multi-modal ranking model to optimize the personalization of the generated images instead of forcing depend on consistency. Extensive experiments and human evaluations on three real-world datasets demonstrate that RAGAR achieves significant improvements in both personalization and semantic metrics compared to five baselines.

Lifelong sequential recommendation aims to model users' long-term interests by leveraging their entire interaction history, but the high computational overhead caused by ultra-long sequences poses a core challenge. To address this, existing methods generally adopt the subsequence learning strategies to shorten the input sequence, which can be divided into two categories: (1) Sequence compression methods compress long sequences into multiple subsequence representations through strategies such as uniform segmentation and clustering; (2) Top-k retrieval methods filter the subsequence related to the target item from long sequences via target attention mechanisms or retrieval mechanisms and learn its representation. However, these two types of methods still face challenges in subsequence representation learning: (1) Sequence compression methods struggle to simultaneously balance the high similarity of items within subsequences and smooth temporal continuity (i.e., small temporal intervals between adjacent items), resulting in incorrect learning of subsequence representations; (2) Top-k retrieval methods lose a large amount of effective context information when the length of the retrieved subsequence is much smaller than the original sequence, resulting in incomplete sequence representations. To overcome these challenges, we propose a novel lifelong sequential recommendation method with adaptive subsequence compression and contextual fusion. Specifically, an adaptive subsequence compression module is first designed: it utilizes gradient policy sampling to achieve adaptive segmentation of subsequences, thereby retaining their temporal continuity, and introduces a reward function to enhance the similarity of items within subsequences. Second, a subsequence context fusion module is constructed: it leverages causal cross-attention to fuse recent interactions with their subsequence context and to capture correlations among recent items, thereby learning more complete and accurate sequence representations. We conduct extensive experiments on three public long-sequence recommendation datasets. Experimental results demonstrate that our proposed method consistently outperforms a variety of strong baselines in both predictive accuracy and computational efficiency.

Data augmentation is an effective technique for tackling data sparsity in sequential recommendation (SR). Existing methods generate new data during the model training to improve the performance. However, deploying them on a backbone model requires retraining, architecture modification, or introducing additional modules and learnable parameters. These processes are time-consuming and costly for well-trained models, especially when the model and data scales become large. In this work, we explore the test-time augmentation (TTA) for SR, which augments the input sequences during the inference phase and then fuses the model's predictions to improve final accuracy. It avoids the significant overhead associated with training-time augmentation. We first experimentally examine the potential of existing augmentation operators for TTA and find that the Substitute and Mask consistently achieve better performance. Further analysis reveals that these two operators retain the original sequential pattern while adding appropriate perturbations. Moreover, the random selection of augmentation positions creates suitable augmented samples from both semantic and temporal perspectives. Meanwhile, we find that the fixed operation ratio limits the diversity of augmented data, and the TTA may impair the model's performance on long sequences. In addition, the two operators still face time-consuming similarity-based item selection or interference from mask tokens. Based on the analysis and limitations, we present TNoise and TMask. The former injects uniform noise into the representation, avoiding the computational overhead of item selection. The latter blocks mask tokens from participating in model calculations (TMask-B) or directly removes interactions that should have been replaced with mask tokens (TMask-R). Further, we sample the augmentation ratio from a uniform distribution to improve the data diversity. For short sequences, we introduce a sequence smoothing and lengthening method based on inter-item interpolation. For long sequences, we set a threshold to avoid the negative effects of TTA. Comprehensive experiments demonstrate the effectiveness, efficiency, and generalizability of our method.

Multi-task learning (MTL) leverages a shared model to accomplish multiple tasks and facilitate knowledge transfer. Recent research on task arithmetic-based MTL demonstrates that merging the parameters of independently fine-tuned models can effectively achieve MTL. However, existing merging methods primarily seek a static optimal solution within the original model parameter space, which often results in performance degradation due to the inherent diversity among tasks and potential interferences. To address this challenge, in this paper, we propose a Weight-Ensembling Mixture of Experts (WEMoE) method for multi-task model merging. Specifically, we first identify critical (or sensitive) modules by analyzing parameter variations in core modules of Transformer-based models before and after fine-tuning. Then, our WEMoE statically merges non-critical modules while transforming critical modules into a mixture-of-experts (MoE) structure. During inference, expert modules in the MoE are dynamically merged based on input samples, enabling a more flexible and adaptive merging approach. Building on WEMoE, we further introduce an efficient-and-effective WEMoE (E-WEMoE) method, whose core mechanism involves eliminating non-essential elements in the critical modules of WEMoE and implementing shared routing across multiple MoE modules, thereby significantly reducing both the trainable parameters, the overall parameter count, and computational overhead of the merged model by WEMoE. Experimental results across various architectures and tasks demonstrate that both WEMoE and E-WEMoE outperform state-of-the-art (SOTA) model merging methods in terms of MTL performance, generalization, and robustness.

Sequential recommendation (SR) has received much attention and made promising progress in the past few years due to its high alignment with real recommendation scenarios. It models users' preferences and behavior patterns from their historical behavior sequences and provides personalized recommendations. However, the widespread problem of data sparsity limits the performance of sequential recommendation models. To tackle this, data augmentation (DA) provides a feasible solution by improving the quantity, quality, or diversity of the training samples without the need for additional data collection. In this survey, we present a systematic and timely review of research efforts on data augmentation for sequential recommendation. We start by providing a clear formulation of the problem and task. Then, we develop a unified taxonomy that categorizes existing augmentation methodologies regarding their augmentation objects and principles. Next, we conduct a comparative discussion on the advantages and disadvantages of different categories, supplemented with quantitative performance evaluations, time-complexity analyses, and visual case studies of representative methods, aiming to provide actionable guidance for the selection and development of augmentation methods in real-world scenarios. Finally, we present the future research directions and summarize this survey.

Intent-Aware Sequential Recommendation (ISR) predicts next-item interactions by capturing the implicit intents behind users’ historical behavior sequences. Although existing ISR methods have achieved promising performance, we argue that two smoothness issues remain insufficiently addressed. First, existing methods often overemphasize short-term intents, making it difficult to smoothly balance users’ short- and long-term preferences. Second, hard positive–negative boundaries in intent contrastive learning may introduce misleading contrastive signals and disrupt the semantic continuity of user behaviors. To tackle these issues, we propose Smoothed Intent Learning for Sequential Recommendation (SILRec) , which improves the smoothness of short- and long-term intents modeling as well as intent contrastive samples. Specifically, we design an SILRec encoder with a multi-scale exponential moving average module and a frequency-domain module to model user intents across different temporal and spectral patterns. We further introduce boundary-smoothed contrastive learning based on representation interpolation, which constructs smoother transition regions between intent representations and alleviates the rigidity of hard contrastive sample boundaries. Comprehensive experiments on 6 conventional real-world datasets with 13 competitors demonstrate the effectiveness of SILRec. We further conduct an additional evaluation on the KuaiRec short-video dataset to examine its applicability to more dynamic recommendation scenarios. Additional analyses from multiple perspectives, including controlled noise perturbation, long-tail performance, and intent visualization, provide further evidence for the proposed design. Our codes and datasets are available at https://github.com/syf1844803351/SILRec .

By generating new yet effective data, data augmentation has become a promising method to mitigate the data sparsity problem in sequential recommendation. Existing works focus on augmenting the original data but rarely explore the issue of imbalanced relevance and diversity for augmented data, leading to semantic drift problems or limited performance improvements. In this paper, we propose a novel Balanced data Augmentation Plugin for Sequential Recommendation (BASRec) to generate data that balance relevance and diversity. BASRec consists of two modules: Single-sequence Augmentation and Cross-sequence Augmentation. The former leverages the randomness of the heuristic operators to generate diverse sequences for a single user, after which the diverse and the original sequences are fused at the representation level to obtain relevance. Further, we devise a reweighting strategy to enable the model to learn the preferences based on the two properties adaptively. The Cross-sequence Augmentation performs nonlinear mixing between different sequence representations from two directions. It produces virtual sequence representations that are diverse enough but retain the vital semantics of the original sequences. These two modules enhance the model to discover fine-grained preferences knowledge from single-user and cross-user perspectives. Extensive experiments verify the effectiveness of BASRec. The average improvement is up to 72.0% on GRU4Rec, 33.8% on SASRec, and 68.5% on FMLP-Rec. We demonstrate that BASRec generates data with a better balance between relevance and diversity than existing methods.

Multi-behavior recommendation exploits auxiliary behaviors (e.g., view, cart) to help predict users' potential target behavior (e.g., purchase) on a given item. However, existing works suffer from two issues: (1) They generally consider only a single chain from auxiliary behaviors to the target behavior, referred to as a purchase chain (e.g., view -> cart -> purchase), ignoring other valuable purchase chains (e.g., view ->purchase) that are beneficial for recommendation performance. (2) Most studies presume that interacted items in auxiliary behaviors are good for recommendations, and pay little attention to the negative transfer problem. That is, some auxiliary behaviors may negatively transfer the influence to the modeling of target ones (e.g., items viewed but not purchased). To alleviate these issues, we propose a novel Multiple Purchase Chains (MPC) model with negative transfer elimination for multi-behavior recommendation. Specifically, we construct multiple purchase chains from auxiliary to target behaviors according to users' historical interactions, while the representations of a previous behavior will be fed to initialize the next behavior on the chain. Then, we construct a negative graph for the latter behavior and learn the negative representations of users and items which will be filtered out to eliminate negative transfer. Experimental results on two real datasets outperform the best baseline by 40.97% and 47.26% on average in terms of Recall@10 and NDCG@10 respectively, demonstrating the effectiveness of our method.

Prompt tuning is a promising method to fine-tune a pre-trained language model without retraining its large-scale parameters. Instead, it attaches a soft prompt to the input text, whereby downstream tasks can be well adapted by merely learning the embeddings of prompt tokens. Nevertheless, existing methods still suffer from two challenges: (i) they are hard to balance accuracy and efficiency. A longer (shorter) soft prompt generally leads to a better (worse) accuracy but at the cost of more (less) training time. (ii) The performance may not be consistent when adapting to different downstream tasks. We attribute it to the same embedding space but responsible for different requirements of downstream tasks. To address these issues, we propose an Efficient Prompt Tuning method (EPT) by multi-space projection and prompt fusion. Specifically, it decomposes a given soft prompt into a shorter prompt and two low-rank matrices, significantly reducing the training time. Accuracy is also enhanced by leveraging low-rank matrices and the short prompt as additional knowledge sources to enrich the semantics of the original short prompt. In addition, we project the soft prompt into multiple subspaces to improve the performance consistency, and then adaptively learn the combination weights of different spaces through a gating network. Experiments on 13 natural language processing downstream tasks show that our method significantly and consistently outperforms 11 comparison methods with the relative percentage of improvements up to 12.9%, and training time decreased by 14%.

Involving collaborative information in Large Language Models (LLMs) is a promising technique for adapting LLMs for recommendation. Existing methods achieve this by concatenating collaborative features with text tokens into a unified sequence input and then fine-tuning to align these features with LLM's input space. Although effective, in this work, we identify two limitations when adapting LLMs to recommendation tasks, which hinder the integration of general knowledge and collaborative information, resulting in sub-optimal recommendation performance. (1) Fine-tuning LLM with recommendation data can undermine its inherent world knowledge and fundamental competencies, which are crucial for interpreting and inferring recommendation text. (2) Incorporating collaborative features into textual prompts disrupts the semantics of the original prompts, preventing LLM from generating appropriate outputs. In this paper, we propose a new paradigm, Collaborative LoRA (CoRA), with a collaborative query generator. Rather than input space alignment, this method aligns collaborative information with LLM's parameter space, representing them as incremental weights to update LLM's output. This way, LLM perceives collaborative information without altering its general knowledge and text inference capabilities. Specifically, we employ a collaborative filtering model to extract user and item embeddings and inject them into a set number of learnable queries. We then convert collaborative queries into collaborative weights with low-rank properties and merge the collaborative weights into LLM's weights, enabling LLM to perceive the collaborative signals and generate personalized recommendations without fine-tuning or extra collaborative tokens in prompts. Extensive experiments confirm that CoRA effectively integrates collaborative information into LLM, enhancing recommendation performance.

Haoyu Xu, Pengxiang Lan, Enneng Yang, Guibing Guo, Jianzhe Zhao, Linying Jiang, Xingwei Wang. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Multimodal recommendation aims to model user and item representations comprehensively with the involvement of multimedia content for effective recommendations. Existing research has shown that it is beneficial for recommendation performance to combine (user- and item-) ID embeddings with multimodal salient features, indicating the value of IDs. However, there is a lack of a thorough analysis of the ID embeddings in terms of semantics in the literature. In this paper, we revisit the value of ID embeddings for multimodal recommendation and conduct a thorough study regarding its semantics, which we recognize as subtle features of content and structure. Based on our findings, we propose a novel recommendation model by incorporating ID embeddings to enhance the salient features of both content and structure. Extensive experiments on three real-world datasets (Baby, Sports, and Clothing) demonstrate the superiority of our method over state-of-the-art multimodal recommendation methods and the effectiveness of fine-grained ID embeddings.

Pengxiang Lan, Haoyu Xu, Enneng Yang, Yuliang Liang, Guibing Guo, Jianzhe Zhao, Xingwei Wang. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Model merging integrates multiple expert models with diverse capabilities into a unified framework, facilitating collaborative learning. However, most existing methods assume simultaneous access to all models, which is often impractical in real-world scenarios where models are received sequentially. While some studies have investigated continual model merging (CMM)--which involves sequentially merging multiple models--the challenge of balancing prior knowledge (stability) and incorporating new tasks (plasticity) remains unresolved. This paper, for the first time, formally defines the stability and plasticity of CMM from the perspective of orthogonal projection. Subsequently, we analyze the relationships among the spaces spanned by task data, historical gradients, and accumulated gradients. Building on this, we propose a data-free \textbf{D}ual \textbf{O}rthogonal \textbf{P}rojection (DOP) method, which eliminates data dependence and mitigates interference between the merged model and models for old and new tasks by projecting their parameter differences onto their respective approximate data spaces. Finally, to solve potential conflicts between stability and plasticity, we reformulate DOP as a multi-objective optimization problem and employ a multi-gradient descent algorithm to obtain a Pareto-optimal solution. Extensive experiments across multiple architectures and task configurations validate that our approach significantly outperforms state-of-the-art CMM methods.

Data augmentation has become a promising method of mitigating data sparsity in sequential recommendation.Existing methods generate new yet effective data during model training to improve performance.However, deploying them requires retraining, architecture modification, or introducing additional learnable parameters.These steps are time-consuming and costly for well-trained models, especially when the model scale becomes large.In this work, we explore the test-time augmentation (TTA) for sequential recommendation, which augments the inputs during the model inference and then aggregates the model's predictions for augmented data to improve final accuracy.It avoids significant time and cost overhead from the previously mentioned steps.We first experimentally disclose the potential of existing augmentation operators for TTA and find that the Mask and Substitute consistently achieve better performance.Further analysis reveals that these two operators are effective because they retain the original sequential pattern while adding appropriate perturbations.Meanwhile, we argue that these two operators still face time-consuming item selection or interference information from mask tokens.Based on the analysis and limitations, we present TNoise and TMask.The former injects uniform noise into the original representation, avoiding the computational overhead of item selection.The latter blocks

Logical reasoning-based recommendation methods employ logical rules to mitigate the adverse effects of noise items in short interaction sequences on recommendation accuracy.However, there are two problems with existing methods: 1) As the length of the interaction sequence increases, introducing more noise items exacerbates the negative impact on logical reasoning, thereby reducing the accuracy of these methods.2) They are often dominated by the user's single primary interest, which prevents simultaneous consideration of users' multiple-aspect interests in long sequences.To address these issues, we propose a novel dEnoising Multi-Interestaware Logical rEasoning (EMILE) method for long-sequence recommendation.Specifically, we design a logical rule-based interest extractor that enhances the importance of preferred items in constructing user interests while minimizing the negative impact of disliked items.This extractor effectively mitigates the adverse effects of noise items in long interaction sequences.Furthermore, we propose a novel multi-interest learning strategy that optimizes two new objective functions-interest probability distribution contrastive loss and interest logical reasoning contrastive loss-to ensure the model simultaneously considers multiple-aspect interests.These two objective functions require that the target item is more * Corresponding authors.

Graph Neural Networks (GNNs)-based recommendation algorithms typically assume that training and testing data are drawn from independent and identically distributed (IID) spaces. However, this assumption often fails in the presence of out-of-distribution (OOD) data, resulting in significant performance degradation. In this study, we construct a Structural Causal Model (SCM) to analyze interaction data, revealing that environmental confounders (e.g., the COVID-19 pandemic) lead to unstable correlations in GNN-based models, thus impairing their generalization to OOD data. To address this issue, we propose a novel approach, graph representation learning via causal diffusion (CausalDiffRec) for OOD recommendation. This method enhances the model's generalization on OOD data by eliminating environmental confounding factors and learning invariant graph representations. Specifically, we use backdoor adjustment and variational inference to infer the real environmental distribution, thereby eliminating the impact of environmental confounders. This inferred distribution is then used as prior knowledge to guide the representation learning in the reverse phase of the diffusion process to learn the invariant representation. In addition,we provide a theoretical derivation that proves optimizing the objective function of CausalDiffRec can encourage the model to learn environment-invariant graph representations, thereby achieving excellent generalization performance in recommendations under distribution shifts. Our extensive experiments validate the effectiveness of CausalDiffRec in improving the generalization of OOD data, and the average improvement is up to 10.69% on Food, 18.83% on KuaiRec, 22.41% on Yelp2018, and 11.65% on Douban datasets.

The distributionally robust optimization (DRO)-based graph neural network methods improve recommendation systems' out-of-distribution (OOD) generalization by optimizing the model's worst-case performance. However, these studies fail to consider the impact of noisy samples in the training data, which results in diminished generalization capabilities and lower accuracy. Through experimental and theoretical analysis, this paper reveals that current DRO-based graph recommendation methods assign greater weight to noise distribution, leading to model parameter learning being dominated by it. When the model overly focuses on fitting noise samples in the training data, it may learn irrelevant or meaningless features that cannot be generalized to OOD data. To address this challenge, we design a Distributionally Robust Graph model for OOD recommendation (DRGO). Specifically, our method first employs a simple and effective diffusion paradigm to alleviate the noisy effect in the latent space. Additionally, an entropy regularization term is introduced in the DRO objective function to avoid extreme sample weights in the worst-case distribution. Finally, we provide a theoretical proof of the generalization error bound of DRGO as well as a theoretical analysis of how our approach mitigates noisy sample effects, which helps to better understand the proposed framework from a theoretical perspective. We conduct extensive experiments on four datasets to evaluate the effectiveness of our framework against three typical distribution shifts, and the results demonstrate its superiority in both independently and identically distributed distributions (IID) and OOD.

The goal of continual learning (CL) is to learn from a series of continuously arriving new tasks without forgetting previously learned old tasks. To avoid catastrophic forgetting of old tasks, orthogonal gradient projection (OGP) based CL methods constrain the gradients of new tasks to be orthogonal to the space spanned by old tasks. This strict gradient constraint will limit the learning ability of new tasks, resulting in lower performance on new tasks. In this paper, we first establish a unified framework for OGP-based CL methods. We then revisit OGP-based CL methods from a new perspective on the loss landscape, where we find that when relaxing projection constraints to improve performance on new tasks, the unflatness of the loss landscape can lead to catastrophic forgetting of old tasks. Based on our findings, we propose a new Dual Flatness-aware OGD framework that optimizes the flatness of the loss landscape from both data and weight levels. Our framework consists of three modules: data and weight perturbation, flatness-aware optimization, and gradient projection. Specifically, we first perform perturbations on the task's data and current model weights to make the task's loss reach the worst-case. Next, we optimize the loss and loss landscape on the original data and the worst-case perturbed data to obtain a flatness-aware gradient. Finally, the flatness-aware gradient will update the network in directions orthogonal to the space spanned by the old tasks. Extensive experiments on four benchmark datasets show that the framework improves the flatness of the loss landscape and performance on new tasks, and achieves state-of-the-art (SOTA) performance on average accuracy across all tasks.

Recommendation unlearning is an emerging task to erase the influences of user-specified data from a trained recommendation model. Most existing research follows the paradigm of partitioning the original dataset into multi-fold and then retraining corresponding sub-models while those influences are totally removed. Despite the effectiveness, two key problems remain unexplored: (i) Existing work becomes inefficient and computationally expensive to retrain all sub-models, especially when facing large amounts of unlearning data. (ii) User preferences are dynamically changing. If users express negative opinions on some interacted items they used to prefer, how can we adaptively erase the outdated preferences behind such transformation from the trained model? Although these unlearning data contain outdated information, there is still a lot of helpful knowledge worth preserving. Existing methods ignore this preservation during unlearning and may remove all the knowledge in the interactions, compromising the final performance. In light of these limitations, we propose a novel unlearning framework called GFEraser, which transforms the unlearning into an efficient guided filtering process to avoid time-consuming retraining and retain beneficial knowledge. Specifically, we develop an intra-user negative sampling strategy to learn the outdated preferences that need to be erased. Under the guidance of differential maximization agreement and attention-based fusion module, the original representations are adaptively filtered and aggregated based on the learned preferences. Besides, we leverage contrastive learning to preserve the invariant user preferences, maintaining the final performance. Finally, we devise a new metric called Ranking Decrease Rate to evaluate the unlearning effect. Experimental results demonstrate that GFEraser can maintain reliable recommendation performance while achieving efficient outdated preferences unlearning, up to 37 \(\times\) acceleration.

Preference logical reasoning utilizes user-item interactions (e.g., ratings and reviews) to infer user preferences and discover user decision paths from the knowledge graph to enhance the explainability of item recommendations. However, existing algorithms assume that the ratings and reviews of any item are always consistent, ignoring situations where items with high ratings have negative reviews or items with low ratings but positive reviews. This leads to inaccurate learning of user preferences. In fact, through experimental analysis of two real datasets, we found that on average, about 10% of the interactive data exhibited this inconsistency, that is, items with high ratings but negative reviews appear in the recommendation list. To address this issue, we propose a general preference logical reasoning method based on preference operators. Specifically, we capture the semantic information of users toward the item (its corresponding attributes) in reviews and define two preference operators ( like and dislike ) for the item to correct ambiguous neutral ratings or false ratings that do not reflect true preferences. In the process of preference path reasoning, the like preference operator increases the occurrence probability of liked items, while the dislike preference operator reduces the occurrence probability of disliked items. By fusing the preference operators in the preference path, we obtain consistent user preferences and enhance the explainability of item recommendations. The experimental results on four real datasets demonstrate that our method can effectively improve the performance of all comparison baselines in terms of recommendation accuracy and user decision explainability.

Graph Contrastive Learning (GCL) leverages data augmentation techniques to produce contrasting views, enhancing the accuracy of recommendation systems through learning the consistency between contrastive views. However, existing augmentation methods, such as directly perturbing interaction graph (e.g., node/edge dropout), may interfere with the original connections and generate poor contrasting views, resulting in sub-optimal performance. In this article, we define the views that share only a small amount of information with the original graph due to poor data augmentation as noisy views (i.e., the last 20% of the views with a cosine similarity value less than 0.1 to the original view). We demonstrate through detailed experiments that noisy views will significantly degrade recommendation performance. Further, we propose a model-agnostic Symmetric Graph Contrastive Learning (SGCL) method with theoretical guarantees to address this issue. Specifically, we introduce symmetry theory into graph contrastive learning, based on which we propose a symmetric form and contrast loss resistant to noisy interference. We provide theoretical proof that our proposed SGCL method has a high tolerance to noisy views. Further demonstration is given by conducting extensive experiments on three real-world datasets. The experimental results demonstrate that our approach substantially increases recommendation accuracy, with relative improvements reaching as high as 12.25% over nine other competing models. These results highlight the efficacy of our method. The code is available at https://github.com/user683/SGCL .

Recently, the powerful large language models (LLMs) have been instrumental in propelling the progress of recommender systems (RS).However, while these systems have flourished, their susceptibility to security threats has been largely overlooked.In this work, we reveal that the introduction of LLMs into recommendation models presents new security vulnerabilities due to their emphasis on the textual content of items.We demonstrate that attackers can significantly boost an item's exposure by merely altering its textual content during the testing phase, without requiring direct interference with the model's training process.Additionally, the attack is notably stealthy, as it does not affect the overall recommendation performance and the modifications to the text are subtle, making it difficult for users and platforms to detect.Our comprehensive experiments across four mainstream LLM-based recommendation models demonstrate the superior efficacy and stealthiness of our approach.Our work unveils a significant security gap in LLM-based recommendation systems and paves the way for future research on protecting these systems.

Multi-task learning (MTL) aims to empower a model to tackle multiple tasks simultaneously. A recent development known as task arithmetic has revealed that several models, each fine-tuned for distinct tasks, can be directly merged into a single model to execute MTL without necessitating a retraining process using the initial training data. Nevertheless, this direct addition of models often leads to a significant deterioration in the overall performance of the merged model. This decline occurs due to potential conflicts and intricate correlations among the multiple tasks. Consequently, the challenge emerges of how to merge pre-trained models more effectively without using their original training data. This paper introduces an innovative technique called Adaptive Model Merging (AdaMerging). This approach aims to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data. Specifically, our AdaMerging method operates as an automatic, unsupervised task arithmetic scheme. It leverages entropy minimization on unlabeled test samples from the multi-task setup as a surrogate objective function to iteratively refine the merging coefficients of the multiple models. Our experimental findings across eight tasks demonstrate the efficacy of the AdaMerging scheme we put forth. Compared to the current state-of-the-art (SOTA) task arithmetic merging scheme, AdaMerging showcases a remarkable 11\% improvement in performance. Notably, AdaMerging also exhibits superior generalization capabilities when applied to unseen downstream tasks. Furthermore, it displays a significantly enhanced robustness to data distribution shifts that may occur during the testing phase.

Multi-task learning (MTL) compresses the information from multiple tasks into a unified backbone to improve computational efficiency and generalization. Recent work directly merges multiple independently trained models to perform MTL instead of collecting their raw data for joint training, greatly expanding the application scenarios of MTL. However, by visualizing the representation distribution of existing model merging schemes, we find that the merged model often suffers from the dilemma of representation bias. That is, there is a significant discrepancy in the representation distribution between the merged and individual models, resulting in poor performance of merged MTL. In this paper, we propose a representation surgery solution called ``Surgery" to reduce representation bias in the merged model. Specifically, Surgery is a lightweight task-specific plugin that takes the representation of the merged model as input and attempts to output the biases contained in the representation from the merged model. We then designed an unsupervised optimization objective that updates the Surgery plugin by minimizing the distance between the merged model's representation and the individual model's representation. Extensive experiments demonstrate significant MTL performance improvements when our Surgery plugin is applied to state-of-the-art (SOTA) model merging schemes.

Sequential recommendation aims to provide users with personalized suggestions based on their historical interactions. When training sequential models, padding is a widely adopted technique for two main reasons: 1) The vast majority of models can only handle fixed-length sequences; 2) Batch-based training needs to ensure that the sequences in each batch have the same length. The special value 0 is usually used as the padding content, which does not contain the actual information and is ignored in the model calculations. This common-sense padding strategy leads us to a problem that has never been explored in the recommendation field: Can we utilize this idle input space by padding other content to improve model performance and training efficiency further?

High-throughput DNA sequencing technologies decode tremendous amounts of microbial protein-coding gene sequences. However, accurately assigning protein functions to novel gene sequences remain a challenge. To this end, we developed FunGeneTyper, an extensible framework with two new deep learning models (i.e., FunTrans and FunRep), structured databases, and supporting resources for achieving highly accurate (Accuracy > 0.99, F1-score > 0.97) and fine-grained classification of antibiotic resistance genes (ARGs) and virulence factor genes. Using an experimentally confirmed dataset of ARGs comprising remote homologous sequences as the test set, our framework achieves by-far-the-best performance in the discovery of new ARGs from human gut (F1-score: 0.6948), wastewater (0.6072), and soil (0.5445) microbiomes, beating the state-of-the-art bioinformatics tools and sequence alignment-based (F1-score: 0.0556-0.5065) and domain-based (F1-score: 0.2630-0.5224) annotation approaches. Furthermore, our framework is implemented as a lightweight, privacy-preserving, and plug-and-play neural network module, facilitating its versatility and accessibility to developers and users worldwide. We anticipate widespread utilization of FunGeneTyper (https://github.com/emblab-westlake/FunGeneTyper) for precise classification of protein-coding gene functions and the discovery of numerous valuable enzymes. This advancement will have a significant impact on various fields, including microbiome research, biotechnology, metagenomics, and bioinformatics.

Continual learning (CL) aims to learn new tasks without forgetting previous tasks. However, existing CL methods require a large amount of raw data, which is often unavailable due to copyright considerations and privacy risks. Instead, stakeholders usually release pre-trained machine learning models as a service (MLaaS), which users can access via APIs. This paper considers two practical-yet-novel CL settings: data-efficient CL (DECL-APIs) and data-free CL (DFCL-APIs), which achieve CL from a stream of APIs with partial or no raw data. Performing CL under these two new settings faces several challenges: unavailable full raw data, unknown model parameters, heterogeneous models of arbitrary architecture and scale, and catastrophic forgetting of previous APIs. To overcome these issues, we propose a novel data-free cooperative continual distillation learning framework that distills knowledge from a stream of APIs into a CL model by generating pseudo data, just by querying APIs. Specifically, our framework includes two cooperative generators and one CL model, forming their training as an adversarial game. We first use the CL model and the current API as fixed discriminators to train generators via a derivative-free method. Generators adversarially generate hard and diverse synthetic data to maximize the response gap between the CL model and the API. Next, we train the CL model by minimizing the gap between the responses of the CL model and the black-box API on synthetic data, to transfer the API's knowledge to the CL model. Furthermore, we propose a new regularization term based on network similarity to prevent catastrophic forgetting of previous APIs. Our method performs comparably to classic CL with full raw data on the MNIST and SVHN datasets in the DFCL-APIs setting. In the DECL-APIs setting, our method achieves 0.97×, 0.75× and 0.69× performance of classic CL on the more challenging CIFAR10, CIFAR100, and MiniImageNet, respectively.

Recommender systems are influenced by many confounding factors (i.e., confounders) which result in various biases (e.g., popularity biases) and inaccurate user preference. Existing approaches try to eliminate these biases by inference with causal graphs. However, they assume all confounding factors can be observed and no hidden confounders exist. We argue that many confounding factors (e.g., season) may not be observable from user–item interaction data, resulting inaccurate user preference. In this article, we propose a deconfounded recommender considering unobservable confounders. Specifically, we propose a new causal graph with explicit and implicit feedback, which can better model user preference. Then, we realize a deconfounded estimator by the front-door adjustment, which is able to eliminate the effect of unobserved confounders. Finally, we conduct a series of experiments on two real-world datasets, and the results show that our approach performs better than other counterparts in terms of recommendation accuracy.

Multi-scenario and multi-task recommendation can use various feedback behaviors of users in different scenarios to learn users’ preferences and then make recommendations, which has attracted attention. However, the existing work ignores feature interactions and the fact that a pair of feature interactions will have differing levels of importance under different scenario-task pairs, leading to sub-optimal user preference learning. In this article, we propose a M ulti-scenario and M ulti-task aware F eature I nteraction model, dubbed MMFI , to explicitly model feature interactions and learn the importance of feature interaction pairs in different scenarios and tasks. Specifically, MMFI first incorporates a pairwise feature interaction unit and a scenario-task interaction unit to effectively capture the interaction of feature pairs and scenario-task pairs. Then MMFI designs a scenario-task aware attention layer for learning the importance of feature interactions from coarse-grained to fine-grained, improving the model’s performance on various scenario-task pairs. More specifically, this attention layer consists of three modules: a fully shared bottom module, a partially shared middle module, and a specific output module. Finally, MMFI adapts two sparsity-aware functions to remove some useless feature interactions. Extensive experiments on two public datasets demonstrate the superiority of the proposed method over the existing multi-task recommendation, multi-scenario recommendation, and multi-scenario & multi-task recommendation models.

Sequential recommendation has now been more widely studied, characterized by its well-consistency with real-world recommendation situations. Most existing works model user preference as the transition pattern from the previous item to the next, ignoring the time interval between these two items. However, we find that the time intervals in different sequences may vary significantly and thus result in the ineffectiveness of user modeling due to the issue ofpreference drift. Thus we propose an assumption that a sequence with uniformly distributed time intervals (denoted as uniform sequence) is more beneficial for preference learning than that with greatly varying time intervals. We then conduct an empirical study on four real datasets and the results support this assumption. Therefore, we advocate to augment sequence data from the perspective of time intervals, which is not studied in the literature. Specifically, we design five operators (Ti-Crop, Ti-CateReorder, Ti-Mask, Ti-Substitute, Ti-Insert) to transform the original non-uniform sequence to uniform sequence with the consideration of time intervals. Then, we devise a control strategy to execute data augmentation on item sequences in different lengths and a looseness range to ensure the generalization (or diversity) of generated data. Finally, we implement these improvements on a state-of-the-art model CoSeRec and proposeTimeInterval AwareCoSeRec(TiCoSeRec). Experimental results on four datasets demonstrate that TiCoSeRec achieves significantly better performance than other 11 counterparts recommendation techniques.

Sequential recommendation is an important task to predict the next-item to access based on a sequence of interacted items. Most existing works learn user preference as the transition pattern from the previous item to the next one, ignoring the time interval between these two items. However, we observe that the time interval in a sequence may vary significantly different, and thus result in the ineffectiveness of user modeling due to the issue of preference drift. In fact, we conducted an empirical study to validate this observation, and found that a sequence with uniformly distributed time interval (denoted as uniform sequence) is more beneficial for performance improvement than that with greatly varying time interval. Therefore, we propose to augment sequence data from the perspective of time interval, which is not studied in the literature. Specifically, we design five operators (Ti-Crop, Ti-Reorder, Ti-Mask, Ti-Substitute, Ti-Insert) to transform the original non-uniform sequence to uniform sequence with the consideration of variance of time intervals. Then, we devise a control strategy to execute data augmentation on item sequences in different lengths. Finally, we implement these improvements on a state-of-the-art model CoSeRec and validate our approach on four real datasets. The experimental results show that our approach reaches significantly better performance than the other 9 competing methods. Our implementation is available: https://github.com/KingGugu/TiCoSeRec.

Multi-task learning (MTL) models have demonstrated impressive results in computer vision, natural language processing, and recommender systems. Even though many approaches have been proposed, how well these approaches balance different tasks on each parameter still remains unclear. In this paper, we propose to measure the task dominance degree of a parameter by the total updates of each task on this parameter. Specifically, we compute the total updates by the exponentially decaying Average of the squared Updates (AU) on a parameter from the corresponding task. Based on this novel metric, we observe that many parameters in existing MTL methods, especially those in the higher shared layers, are still dominated by one or several tasks. The dominance of AU is mainly due to the dominance of accumulative gradients from one or several tasks. Motivated by this, we propose a Task-wise Adaptive learning rate approach, AdaTask in short, to separate the accumulative gradients and hence the learning rate of each task for each parameter in adaptive learning rate approaches (e.g., AdaGrad, RMSProp, and Adam). Comprehensive experiments on computer vision and recommender system MTL datasets demonstrate that AdaTask significantly improves the performance of dominated tasks, resulting SOTA average task-wise performance. Analysis on both synthetic and real-world datasets shows AdaTask balance parameters in every shared layer well.

The goal of continual learning (CL) is to continuously learn new tasks without forgetting previously learned old tasks. To alleviate catastrophic forgetting, gradient projection based CL methods require that the gradient updates of new tasks are orthogonal to the subspace spanned by old tasks. This limits the learning process and leads to poor performance on the new task due to the projection constraint being too strong. In this paper, we first revisit the gradient projection method from the perspective of flatness of loss surface, and find that unflatness of the loss surface leads to catastrophic forgetting of the old tasks when the projection constraint is reduced to improve the performance of new tasks. Based on our findings, we propose a Data Augmented Flatness-aware Gradient Projection (DFGP) method to solve the problem, which consists of three modules: data and weight perturbation, flatness-aware optimization, and gradient projection. Specifically, we first perform a flatness-aware perturbation on the task data and current weights to find the case that makes the task loss worst. Next, flatness-aware optimization optimizes both the loss and the flatness of the loss surface on raw and worst-case perturbed data to obtain a flatness-aware gradient. Finally, gradient projection updates the network with the flatness-aware gradient along directions orthogonal to the subspace of the old tasks. Extensive experiments on four datasets show that our method improves the flatness of loss surface and the performance of new tasks, and achieves state-of-the-art (SOTA) performance in the average accuracy of all tasks.

Basket representation plays an important role in the task of next-basket recommendation. However, existing methods generally adopts pooling operations to learn a basket's representation, from which two critical issues can be identified. First, they treat a basket as a set of items independent and identically distributed. We find that items occurring in the same basket have much higher correlations than those randomly selected by conducting data analysis on a real dataset. Second, although some works have recognized the importance of items repeatedly purchased in multiple baskets, they ignore the correlations among the repeated items in a same basket, whose importance is shown by our data analysis. In this paper, we propose a novel Basket Representation Learning (BRL) model by leveraging the correlations among intra-basket items. Specifically, we first connect all the items (in a basket) as a hyperedge, where the correlations among different items can be well exploited by hypergraph convolution operations. Meanwhile, we also connect all the repeated items in the same basket as a hyperedge, whereby their correlations can be further strengthened. We generate a negative (positive) view of the basket by data augmentation on repeated (non-repeated) items, and apply contrastive learning to force more agreements on repeated items. Finally, experimental results on three real datasets show that our approach performs better than eight baselines in ranking accuracy.

Dataset condensation (DC) distills a large real-world dataset into a small synthetic dataset, with the goal of training a network from scratch on the latter that performs similarly to the former. State-of-the-art (SOTA) DC methods have achieved satisfactory results through techniques such as accuracy, gradient, training trajectory, or distribution matching. However, these works all perform matching in the high-dimension pixel spaces, ignoring that natural images are usually locally connected and have lower intrinsic dimensions, resulting in low condensation efficiency. In this work, we propose a simple-yet-efficient dataset condensation plugin that matches the raw and synthetic datasets in a low-dimensional manifold. Specifically, our plugin condenses raw images into two low-rank matrices instead of parameterized image matrices. Our plugin can be easily incorporated into existing DC methods, thereby containing richer raw dataset information at limited storage costs to improve the downstream applications' performance. We verify on multiple public datasets that when the proposed plugin is combined with SOTA DC methods, the performance of the network trained on synthetic data is significantly improved compared to traditional DC methods. Moreover, when applying the DC methods as a plugin to continual learning tasks, we observed that our approach effectively mitigates catastrophic forgetting of old tasks under limited memory buffer constraints and avoids the problem of raw data privacy leakage.

Recently, Memory-based Neural Recommenders (MNR) have demonstrated superior predictive accuracy in the task of sequential recommendations, particularly for modeling long-term item dependencies. However, typical MNR requires complex memory access operations, i.e., both writing and reading via a controller (e.g., RNN) at every time step. Those frequent operations will dramatically increase the network training time, resulting in the difficulty in being deployed on industrial-scale recommender systems. In this paper, we present a novel generalChunkframework to accelerate MNR significantly. Specifically, our framework divides proximal information units into chunks, and performs memory access at certain time steps, whereby the number of memory operations can be greatly reduced. We investigate two ways to implement effective chunking, i.e., PEriodic Chunk (PEC) and Time-Sensitive Chunk (TSC), to preserve and recover important recurrent signals in the sequence. Since chunk-accelerated MNR models take into account more proximal information units than that from a single timestep, it can alleviate the influence of noise in the user-item interaction sequence to a large extent, and thus improve the stability of MNR. In this way, the proposed chunk mechanism can lead to not only faster training and prediction, but even slightly better results. The experimental results on three real-world datasets (weishi, ml-10M and ml-latest) show that our chunk framework notably reduces the running time (e.g., with up to 7x for training & 10x for inference on ml-latest) of MNR, and meantime achieves competitive performance.

Negative sampling plays an important role in ranking-based recommender models. However, most existing sampling methods cannot generate informative item pairs with positive and negative instances due to two limitations: 1) they merely treat observed items as positive instances, ignoring the existence of potential positive items (i.e., nonobserved items users may prefer) and the probability of observed but noisy items and 2) they fail to capture the relationship between positive and negative items during negative sampling, which may cause the unexpected selection of potential positive items. In this article, we introduce a dynamic sampling strategy to search informative item pairs. Specifically, we first sample a positive instance from all the items by leveraging the overall features of user's observed items. Then, we strategically select a negative instance by considering its correlation with the sampled positive one. Formally, we propose an item pair generative adversarial network named IPGAN, where our sampling strategy is realized in two generative models for positive and negative instances, respectively. In addition, IPGAN can also ensure that the sampled item pairs are informative relative to the ground truth by a discriminative model. What is more, we propose a batch-training approach to further enhance both user and item modeling by alleviating the special bias (noise) from different users. This approach can also significantly accelerate the process of model training compared with classical GAN method for recommendation. Experimental results on three real data sets show that our approach outperforms other state-of-the-art approaches in terms of recommendation accuracy.

The consistency of a response to a given post at the semantic level and emotional level is essential for a dialogue system to deliver humanlike interactions. However, this challenge is not well addressed in the literature, since most of the approaches neglect the emotional information conveyed by a post while generating responses. This article addresses this problem and proposes a unified end-to-end neural architecture, which is capable of simultaneously encoding the semantics and the emotions in a post and leveraging target information to generate more intelligent responses with appropriately expressed emotions. Extensive experiments on real-world data demonstrate that the proposed method outperforms the state-of-the-art methods in terms of both content coherence and emotion appropriateness.

Paper recommendation is a research topic to provide users with personalized papers of interest. However, most existing approaches equally treat title and abstract as the input to learn the representation of a paper, ignoring their semantic relationship. In this paper, we regard the abstract as a sequence of sentences, and propose a two-level attentive neural network to capture: (1) the ability of each word within a sentence to reflect if it is semantically close to the words within the title. (2) the extent of each sentence in the abstract relative to the title, which is often a good summarization of the abstract document. Specifically, we propose a Long-Short Term Memory (LSTM) network with attention to learn the representation of sentences, and integrate a Gated Recurrent Unit (GRU) network with a memory network to learn the long-term sequential sentence patterns of interacted papers for both user and item (paper) modeling. We conduct extensive experiments on two real datasets, and show that our approach outperforms other state-of-the-art approaches in terms of accuracy.

Current recommendation systems usually use image as a type of side information to enhance recommendation systems. However, there are few models to analyze and apply the effectiveness of different categories of images. In order to effectively combine multi-category image information in the recommendation system, we propose a novel deep network model with different categories of images as side information for recommendation, in which we use convolutional attention to analyze the importance of images within different image categories. The convolutional layers of the attention module are used to evaluate the user's visual preferences for images in different categories. The prediction model uses the cosine similarity of user factors and restaurant factors incorporating visual information. Visual features of the restaurants are extracted from multi-category images by a pre-trained neural network. We apply our model to two real-world restaurant recommendation data sets. Experimental results show that the performance of our model is better than models without or with only one category visual information.

Recently, GAN-based collaborative filtering methods have gained increasing attention in recommendation tasks which can learn remarkable user and item representation. However, these existing GAN-based methods mainly suffer from two limitations: (1) Their trainings are not comprehensive given the fact that the discriminator may be trained misleadingly and over-early converging since the generator may accidentally sample real items as fake ones, resulting in the emergence of contradicting labels for the same items. (2) They fail to consider implicit friends (users with the same interests.), leading to severe limitations of recommendation performance. In this paper, we propose BiGAN, an innovative bidirectional adversarial recommendation model which can alleviate the limitations mentioned above in recommendation tasks. It consists of two GANs, namely ForwardGAN and BackwardGAN. Specifically, ForwardGAN learns to generate a group of possible interacted items given a specific user, it aims to ensure that the discriminator Df can be trained effectively. Furthermore, BackwardGAN fully exploits implicit friends with similar behaviors, then propagates them back to ForwardGAN, where a similarity exploration strategy is implemented to gain more outstanding user representation. Therefore, two GANs are trained jointly in a circle, where the augment of one GAN will enhance another one, leading to the promising user and item representation. In the experimental part, we demonstrate that our model is superior to other state-of-the-art recommenders.

Session-based recommender systems have attracted much attention recently. To capture the sequential dependencies, existing methods resort either to data augmentation techniques or left-to-right style autoregressive training. Since these methods are aimed to model the sequential nature of user behaviors, they ignore the future data of a target interaction when constructing the prediction model for it. However, we argue that the future interactions after a target interaction, which are also available during training, provide valuable signal on user preference and can be used to enhance the recommendation quality.

In the field of sequential recommendation, deep learning--(DL) based methods have received a lot of attention in the past few years and surpassed traditional models such as Markov chain-based and factorization-based ones. However, there is little systematic study on DL-based methods, especially regarding how to design an effective DL model for sequential recommendation. In this view, this survey focuses on DL-based sequential recommender systems by taking the aforementioned issues into consideration. Specifically, we illustrate the concept of sequential recommendation, propose a categorization of existing algorithms in terms of three types of behavioral sequences, summarize the key factors affecting the performance of DL-based models, and conduct corresponding evaluations to showcase and demonstrate the effects of these factors. We conclude this survey by systematically outlining future directions and challenges in this field.

The consistency of a response to a given post at semantic-level and emotional-level is essential for a dialogue system to deliver human-like interactions. However, this challenge is not well addressed in the literature, since most of the approaches neglect the emotional information conveyed by a post while generating responses. This article addresses this problem by proposing a unified end-to-end neural architecture, which is capable of simultaneously encoding the semantics and the emotions in a post for generating more intelligent responses with appropriately expressed emotions. Extensive experiments on real-world data demonstrate that the proposed method outperforms the state-of-the-art methods in terms of both content coherence and emotion appropriateness.

Sequential recommendation systems have become a research hotpot recently to suggest users with the next item of interest (to interact with). However, existing approaches suffer from two limitations: (1) The representation of an item is relatively static and fixed for all users. We argue that even a same item should be represented distinctively with respect to different users and time steps. (2) The generation of a prediction for a user over an item is computed in a single scale (e.g., by their inner product), ignoring the nature of multi-scale user preferences. To resolve these issues, in this paper we propose two enhancing building blocks for sequential recommendation. Specifically, we devise a Dynamic Item Block (DIB) to learn dynamic item representation by aggregating the embeddings of those who rated the same item before that time step. Then, we come up with a Prediction Enhancing Block (PEB) to project user representation into multiple scales, based on which many predictions can be made and attentively aggregated for enhanced learning. Each prediction is generated by a softmax over a sampled itemset rather than the whole item space for efficiency. We conduct a series of experiments on four real datasets, and show that even a basic model can be greatly enhanced with the involvement of DIB and PEB in terms of ranking accuracy. The code and datasets can be obtained from https://github.com/ouououououou/DIB-PEB-Sequential-RS

Trust-aware recommender systems have received much attention recently for their abilities to capture the influence among connected users. However, they suffer from the efficiency issue due to large amount of data and time-consuming real-valued operations. Although existing discrete collaborative filtering may alleviate this issue to some extent, it is unable to accommodate social influence. In this paper we propose a discrete trust-aware matrix factorization (DTMF) model to take dual advantages of both social relations and discrete technique for fast recommendation. Specifically, we map the latent representation of users and items into a joint hamming space by recovering the rating and trust interactions between users and items. We adopt a sophisticated discrete coordinate descent (DCD) approach to optimize our proposed model. In addition, experiments on two real-world datasets demonstrate the superiority of our approach against other state-of-the-art approaches in terms of ranking accuracy and efficiency.

Jointing visual-semantic embeddings (VSE) have become a research hotpot for the task of image annotation, which suffers from the issue of semantic gap, i.e., the gap between images' visual features (low-level) and labels' semantic features (high-level). This issue will be even more challenging if visual features cannot be retrieved from images, that is, when images are only denoted by numerical IDs as given in some real datasets. The typical way of existing VSE methods is to perform a uniform sampling method for negative examples that violate the ranking order against positive examples, which requires a time-consuming search in the whole label space. In this paper, we propose a fast adaptive negative sampler that can work well in the settings of no figure pixels available. Our sampling strategy is to choose the negative examples that are most likely to meet the requirements of violation according to the latent factors of images. In this way, our approach can linearly scale up to large datasets. The experiments demonstrate that our approach converges 5.02x faster than the state-of-the-art approaches on OpenImages, 2.5x on IAPR-TCI2 and 2.06x on NUS-WIDE datasets, as well as better ranking accuracy across datasets.

CBOW (Continuous Bag-Of-Words) is one of the most commonly used techniques to generate word embeddings in various NLP tasks. However, it fails to reach the optimal performance due to uniform involvements of positive words and a simple sampling distribution of negative words. To resolve these issues, we propose OptRank to optimize word ranking and approximate negative sampling for bettering word embedding. Specifically, we first formalize word embedding as a ranking problem. Then, we weigh the positive words by their ranks such that highly ranked words have more importance, and adopt a dynamic sampling strategy to select informative negative words. In addition, an approximation method is designed to efficiently compute word ranks. Empirical experiments show that OptRank consistently outperforms its counterparts on a benchmark dataset with different sampling scales, especially when the sampled subset is small. The code and datasets can be obtained from https://github.com/ouououououou/OptRank.

In this paper, we propose a new factorization model that combines multi-view visual feature information with the implicit feedback data for prediction and ranking. The visual information is integrated into a collaborative filtering framework. The visual features of images are extracted by using a deep neural network. In order to conduct personalized recommendation better, the multi-view visual features are fused through user related weights. The user related weights reflect the personalized visual preference for items. They are different and independent between users. Experimental results show that our model with multi-view visual information achieves the better performance than models without or with only single-view visual information.

Recommendation system has became an important component in many real applications, ranging from e-commerce, music app to video-sharing site and on-line book store. The key of a successful recommendation system lies in the accurate user/item profiling. With the advent of web 2.0, quite a lot of multimodal information has been accumulated, which provides us with the opportunity to profile users in a more comprehensive manner. However, directly integrating multimodal information into recommendation system is not a trivial task, because they may be either homogenous or heterogeneous, which requires more advanced method for both fusion and alignment.

Personalized recommendation for online service systems aims to predict potential demand by analysing user preference. User preference can be inferred from heterogeneous implicit feedback (i.e. various user actions) especially when explicit feedback (i.e. ratings) is not available. However, most methods either merely focus on homogeneous implicit feedback (i.e. target action), e.g., purchase in shopping websites and forward in Twitter, or dispose heterogeneous implicit feedback without the investigation of its speciality. In this paper, we adopt two typical actions in online service systems, i.e., view and like , as auxiliary feedback to enhance recommendation performance, whereby we propose a Bayesian personalized ranking method for heterogeneous implicit feedback (BPRH). Specifically, items are first classified into different types according to the actions they received. Then by analysing the co-occurrence of different types of actions, which is one of the fundamental speciality of heterogeneous implicit feedback systems, we quantify their correlations, based on which the difference of users’ preference among different types of items is investigated. An adaptive sampling strategy is also proposed to tackle the unbalanced correlation among different actions. Extensive experimentation on three real-world datasets demonstrates that our approach significantly outperforms state-of-the-art algorithms.

Feature-based matrix factorization techniques such as Factorization Machines (FM) have been proven to achieve impressive accuracy for the rating prediction task. However, most common recommendation scenarios are formulated as a top-N item ranking problem with implicit feedback (e.g., clicks, purchases)rather than explicit ratings. To address this problem, with both implicit feedback and feature information, we propose a feature-based collaborative boosting recommender called BoostFM, which integrates boosting into factorization models during the process of item ranking. Specifically, BoostFM is an adaptive boosting framework that linearly combines multiple homogeneous component recommenders, which are repeatedly constructed on the basis of the individual FM model by a re-weighting scheme. Two ways are proposed to efficiently train the component recommenders from the perspectives of both pairwise and listwise Learning-to-Rank (L2R). The properties of our proposed method are empirically studied on three real-world datasets. The experimental results show that BoostFM outperforms a number of state-of-the-art approaches for top-N recommendation.

In this paper, we propose a decentralized identity management system based on Blockchain. The function of the system mainly includes identity authentication and reputation management. The technical advantages of the Blockchain makes the data in the system safe and credible. In addition, we use smart contracts to write system rules to ensure the reliability of user information. We bind the user's entity information with the public key address and determine the true identity of a virtual user on the Blockchain. We use the token to represent the reputation which is shown to be an effective reputation model, making the participants in the system prefer to maintain and manage their personal reputation. Our system makes it possible for users to securely manage their identity and reputation on the Internet.

Our data analysis on real-world datasets shows that user preferences are intimately related with item categories, implying the non-negligible of category information for effective recommendation. Thus, in this paper, step by step we propose a unified item-category latent factor model by considering user-category, item-category and category-category interactions. Our approach can be applied to both the situations where an item belongs to either a single category (one-to-one) or multiple categories (one-to-many). Finally, empirical studies on the real-world datasets demonstrate the superiority of our approach in comparison with other counterparts.

Web service composition is a process to compose homogenous or heterogeneous services together in order to create value-added services. Many non-functional features including QoS and user preferences have been adopted to guide such a process. However, two issues are observed: (1) the expressiveness of user preference is subject to quantitative preferences without proper use of qualitative preferences; (2) a highly preferred composite service may not be trustworthy, or a highly trustworthy composite service may not be preferable. The existing studies concentrate on either user preference or service trust, and fail to provide a systematic solution to integrate both user preference and service trust together for service compositions. To address these issues, we combine both qualitative and quantitative preferences as well as service trust together in the process of service composition. We investigate the application of heuristic algorithms on multi-objective optimization for the service composition problem. A new hybrid nature inspired intelligent algorithm is also proposed and compared with other popular heuristic algorithms. We aim to obtain optimal web service compositions that can satisfy these (potentially conflicting) constraints as much as possible. Results demonstrate the efficiency and effectiveness of our approach in comparison with other counterparts.

State-of-the-art item recommendation algorithms, which apply Factorization Machines (FM) as a scoring function and pairwise ranking loss as a trainer (PRFM for short), have been recently investigated for the implicit feedback based context-aware recommendation problem (IFCAR). However, good recommenders particularly emphasize on the accuracy near the top of the ranked list, and typical pairwise loss functions might not match well with such a requirement. In this paper, we demonstrate, both theoretically and empirically, PRFM models usually lead to non-optimal item recommendation results due to such a mismatch. Inspired by the success of LambdaRank, we introduce Lambda Factorization Machines (LambdaFM), which is particularly intended for optimizing ranking performance for IFCAR. We also point out that the original lambda function suffers from the issue of expensive computational complexity in such settings due to a large amount of unobserved feedback. Hence, instead of directly adopting the original lambda strategy, we create three effective lambda surrogates by conducting a theoretical analysis for lambda from the top-N optimization perspective. Further, we prove that the proposed lambda surrogates are generic and applicable to a large set of pairwise ranking loss functions. Experimental results demonstrate LambdaFM significantly outperforms state-of-the-art algorithms on three real-world datasets in terms of four standard ranking measures.

Recommending users with preferred point-of-interests (POIs) has become an important task for location-based social networks, which facilitates users' urban exploration by helping them filter out unattractive locations. Although the influence of geographical neighborhood has been studied in the rating prediction task (i.e. regression), few work have exploited it to develop a ranking-oriented objective function to improve top-N item recommendations. To solve this task, we conduct a manual inspection on real-world datasets, and find that each individual's traits are likely to cluster around multiple centers. Hence, we propose a co-pairwise ranking model based on the assumption that users prefer to assign higher ranks to the POIs near previously rated ones. The proposed method can learn preference ordering from non-observed rating pairs, and thus can alleviate the sparsity problem of matrix factorization. Evaluation on two publicly available datasets shows that our method performs significantly better than state-of-the-art techniques for the top-N item recommendation task.

In e-commerce systems, user preference can be inferred from multivariate implicit feedback (i.e., actions). However, most methods merely focus on homogeneous implicit feedback (i.e., purchase). In this paper, we adopt another two typical actions, i.e., view and like, as auxiliaries to enhance purchase recommendation, whereby a trinity Bayesian personalized ranking (TBPR) method is proposed. Specifically, we introduce trinity preference to investigate the difference of users' preference among three types of items: 1) items with purchase action; 2) items with only auxiliary actions; 3) items without any action. Empirical study on the real-world dataset demonstrates that our method significantly outperforms state-of-the-art algorithms.

Although flat item category structure where categories are independent in a same level has been well studied to enhance recommendation performance, in many real applications, item category is often organized in hierarchies to reflect the inherent correlations among categories. In this paper, we propose a novel matrix factorization model by exploiting category hierarchy from the perspectives of users and items for effective recommendation. Specifically, a user (an item) can be influenced (characterized) by her preferred categories (the categories it belongs to) in the hierarchy. We incorporate how different categories in the hierarchy co-influence a user and an item. Empirical results show the superiority of our approach against other counterparts.

We propose TrustSVD, a trust-based matrix factorization technique for recommendations. TrustSVD integrates multiple information sources into the recommendation model in order to reduce the data sparsity and cold start problems and their degradation of recommendation performance. An analysis of social trust data from four real-world data sets suggests that not only the explicit but also the implicit influence of both ratings and trust should be taken into consideration in a recommendation model. TrustSVD therefore builds on top of a state-of-the-art recommendation algorithm, SVD++ (which uses the explicit and implicit influence of rated items), by further incorporating both the explicit and implicit influence of trusted and trusting users on the prediction of items for an active user. The proposed technique is the first to extend SVD++ with social trust information. Experimental results on the four data sets demonstrate that TrustSVD achieves better accuracy than other ten counterparts recommendation techniques.

User-based collaborative filtering , a widely used nearest neighbour-based recommendation technique, predicts an item’s rating by aggregating its ratings from similar users. User similarity is traditionally calculated by cosine similarity or the Pearson correlation coefficient . However, both of these measures consider only the direction of rating vectors, and suffer from a range of drawbacks. To overcome these issues, we propose a novel Bayesian similarity measure based on the Dirichlet distribution, taking into consideration both the direction and length of rating vectors. We posit that not all the rating pairs should be equally counted in order to accurately model user correlation. Three different evidence factors are designed to compute the weights of rating pairs. Further, our principled method reduces correlation due to chance and potential system bias. Experimental results on six real-world datasets show that our method achieves superior accuracy in comparison with counterparts.

Collaborative filtering suffers from the problems of data sparsity and cold start, which dramatically degrade recommendation performance. To help resolve these issues, we propose TrustSVD, a trust-based matrix factorization technique. By analyzing the social trust data from four real-world data sets, we conclude that not only the explicit but also the implicit influence of both ratings and trust should be taken into consideration in a recommendation model. Hence, we build on top of a state-of-the-art recommendation algorithm SVD++ which inherently involves the explicit and implicit influence of rated items, by further incorporating both the explicit and implicit influence of trusted users on the prediction of items for an active user. To our knowledge, the work reported is the first to extend SVD++ with social trust information. Experimental results on the four data sets demonstrate that our approach TrustSVD achieves better accuracy than other ten counterparts, and can better handle the concerned issues.

Web service composition is a process to compose homogenous or heterogeneous services together in order to create value-added services. Many non-functional features including QoS and user preferences have been adopted to guide such a process. However, two issues are observed: (1) the expressiveness of user preference is subject to quantitative preferences without proper use of qualitative preferences, (2) a highly preferred composite service may not be trustworthy, or a highly trustworthy composite service may not be preferable. To address these issues, we combine both qualitative and quantitative preferences as well as service trust together in the process of service composition. We aim to obtain optimal web service compositions that can satisfy these (potentially conflicting) constraints as much as possible. Experimental results demonstrate the efficiency and effectiveness of our approach in comparison with other counterparts.

Trust is one source of information that has been widely adopted to personalize online services for users, such as in product recommendations. However, trust information is usually very sparse or unavailable for most online systems. To narrow this gap, we propose a principled approach that predicts implicit trust from users' interactions, by extending a well-known trust antecedents framework. Specifically, we consider both local and global trustworthiness of target users, and form a personalized trust metric by further taking into account the active user's propensity to trust. Experimental results on two real-world datasets show that our approach works better than contemporary counterparts in terms of trust ranking performance when direct user interactions are limited.

Recommending learning materials for e-learning systems often encounters two issues: how to classify and organize learning materials and how to make effective recommendations. In this paper, we propose a new algorithm to handle these two problems. Specifically, we compile each learning material to concepts according to their relevance which is modeled as the length of a term-weight vector. Then recommendations are generated by taking into account the document’s similarity with some good learning material, the personalized time-aware usefulness of the learning material, the concepts of the learning material as well as their difficulty levels. Experimental results based on a small sample demonstrate the effectiveness of our method in terms of knowledge gain obtained.

In this paper we produce an algorithm for presenting messages to users in social networks that integrates reasoning about the message, the author, the recipient and the social network. Our proposed model was derived on the basis of immersion within three different existing social networking environments, that of Courser a, Reddit, and medical self-help groups such as PatientsLikeMe. We first present three models, each of which is designed to perform well within the context of one specific social network. From here we derive a generalized model which can be effective regardless of social network context. We conclude with a discussion of possible directions for future research, with an emphasis on promoting the use of trust modeling and user modeling, in a view to exploring additional networks and include as well a comparison to competing models within the artificial intelligence literature. Our aim is to offer insights into coping with the massive amount of information that currently resides within our social networks.

Our research aims to tackle the problems of data sparsity and cold start of traditional recommender systems. Insufficient ratings often result in poor quality of recommendations in terms of accuracy and coverage. To address these issues, we propose three different approaches from the perspective of preference modelling. Firstly, we propose to merge the ratings of trusted neighbors and thus form a new rating profile for the active users, based on which better recommendations can be generated. Secondly, we aim to make better use of user ratings and introduce a novel Bayesian similarity measure by taking into account both the direction and length of rating vectors. Thirdly, we propose a new information source called prior ratings based on virtual product experience in virtual reality environments, in order to inherently resolve the concerned problems.

Lack of motivation to provide ratings and eligibility to rate generally only after purchase restrain the effectiveness of recommender systems and contribute to the well-known data sparsity and cold start problems. This paper proposes a new information source for recommender systems, called prior ratings. Prior ratings are based on users' experiences of virtual products in a mediated environment, and they can be submitted prior to purchase. A conceptual model of prior ratings is proposed, integrating the environmental factor presence whose effects on product evaluation have not been studied previously. A user study conducted in website and virtual store modalities demonstrates the validity of the conceptual model, in that users are more willing and confident to provide prior ratings in virtual environments.

Similarity measurement is essential in many preference-based personalized applications such as collaborative recommendation and service selection. Up to date, current researches have mainly focused on the measurements for quantitative preference rather than qualitative preference, although the latter has attracted much attention recently. Only a very few methods to measure user similarity are proposed. This paper aims to fill in this gap by proposing an intuitive similarity measurement for conditional qualitative preference which is represented by CP-nets. Experimental results based on two expanded real-life datasets demonstrate that our similarity measurement is not only able to correctly reflect user's preference changes, but also effective to identify similar users.

PGP Web of Trust where users can sign digital signatures on public key certificates of other users has been successfully applied in securing emails and files transmitted over the Internet. However, its rigorous restrictions on utilizable trust relationships and acceptable signatures limit its performance. In this paper, we first make some modification and extension to PGP Web of Trust by relaxing those constraints. In addition, we propose a novel method to further expand trusted neighborhood of users by merging the signatures of the trusted neighbors and finding the similar users based on the merged signature set. Confirmed by the experiments carried out in different simulated real-life scenarios, our method applied to both the modified and extended PGP methods can improve their performance. With the expansion of trusted neighborhood, the performance of the original PGP Web of Trust is also improved considerably.