DNNs have gained widespread adoption in feature interaction recommendation models. However, there has been a longstanding debate on their roles. On one hand, some works claim that DNNs possess the ability to implicitly capture high-order feature interactions. Conversely, recent studies have highlighted the limitations of DNNs in effectively learning dot products, specifically second-order interactions, let alone higher-order interactions. In this paper, we present a novel perspective to understand the effectiveness of DNNs: their impact on the dimensional robustness of the representations. In particular, we conduct extensive experiments involving both parallel DNNs and stacked DNNs. Our evaluation encompasses an overall study of complete DNN on two feature interaction models, alongside a fine-grained ablation analysis of components within DNNs. Experimental results demonstrate that both parallel and stacked DNNs can effectively mitigate the dimensional collapse of embeddings. Furthermore, a gradient-based theoretical analysis, supported by empirical evidence, uncovers the underlying mechanisms of dimensional collapse. The code is accessible for reproduction. https://github.com/USTC-StarTeam/Dimensional-Collapse-Analysis

In the era of data-centric AI, the focus of recommender systems has shifted from model-centric innovations to data-centric approaches. The success of modern AI models is built on large-scale datasets, but this also results in significant training costs. Dataset distillation has emerged as a key solution, condensing large datasets to accelerate model training while preserving model performance. However, condensing discrete and sequentially correlated user-item interactions, particularly with extensive item sets, presents considerable challenges. This paper introduces TD3, a novel Tucker Decomposition based Dataset Distillation method within a meta-learning framework, designed for sequential recommendation. TD3 distills a fully expressive synthetic sequence summary from original data. To efficiently reduce computational complexity and extract refined latent patterns, Tucker decomposition decouples the summary into four factors: synthetic user latent factor, temporal dynamics latent factor, shared item latent factor, and a relation core that models their interconnections. Additionally, a surrogate objective in bi-level optimization is proposed to align feature spaces extracted from models trained on both original data and synthetic sequence summary beyond the naive performance matching approach. In the inner-loop, an augmentation technique allows the learner to closely fit the synthetic summary, ensuring an accurate update of it in the outer-loop. To accelerate the optimization process and address long dependencies, RaT-BPTT is employed for bi-level optimization. Experiments and analyses on multiple public datasets have confirmed the superiority and cross-architecture generalizability of the proposed designs. Codes are released at https://github.com/USTC-StarTeam/TD3.

Recently, the dual-target cross-domain recommendation has been an emerging research problem, which aims to improve the performances of both source and target domains by transferring the preferences of overlapping users. Most of the existing work adopted a coarse-grained manner to detach general users’ preferences and associate them with domain-specific information for enhancing user representation learning, which fails to depict the differences in users’ diverse preferences and aggregate relevant preferences with improper propagation. To this end, in this article, we propose a multi-factor user representation pre-training framework, dubbed MF-GSLAE, with a focus on fine-grained preference learning and transferring. Specifically, we first propose a fine-grained factor representation pre-training paradigm. It projects the behavior records of both domains into several subspaces and introduces a compactness regularization to generate multiple fine-grained preference factors. Furthermore, we propose a multi-factor graph structure learning method within linear complexity to efficiently construct preference connections on different scales of users, which could aggregate the intrinsic relationship of user preferences in immediate embedding spaces to capture high-order information. Following the pre-training, we subsequently design a factor selection module with the bootstrapping mechanism to adaptively choose the corresponding domain-related preferences and transfer domain-shared information through partial overlapping factors for addressing the negative transfer problem. Finally, the optimization objectives of both domains are formalized in a multi-task learning framework and derive the learned user representation in an end-to-end training manner. Extensive experimental results on several publicly available datasets have not only demonstrated the effectiveness of the learned user representations with the comparison of state-of-the-art baselines but also indicated the interpretability and robustness. The code of our work is publicly available at https://github.com/USTC-StarTeam/MF-GSLAE .

The sequential recommender (SR) system is a crucial component of modern recommender systems, as it aims to capture the evolving preferences of users. Significant efforts have been made to enhance the capabilities of SR systems. These methods typically follow the model-centric paradigm, which involves developing effective models based on fixed datasets. However, this approach often overlooks potential quality issues and flaws inherent in the data. Driven by the potential of data-centric AI, we propose a novel data-centric paradigm for developing an ideal training dataset using a model-agnostic dataset regeneration framework called DR4SR. This framework enables the regeneration of a dataset with exceptional cross-architecture generalizability. Additionally, we introduce the DR4SR+ framework, which incorporates a model-aware dataset personalizer to tailor the regenerated dataset specifically for a target model. To demonstrate the effectiveness of the data-centric paradigm, we integrate our framework with various model-centric methods and observe significant performance improvements across four widely adopted datasets. Furthermore, we conduct in-depth analyses to explore the potential of the data-centric paradigm and provide valuable insights. The code can be found at https://github.com/USTC-StarTeam/DR4SR.

The sequential recommendation system has been widely studied for its promising effectiveness in capturing dynamic preferences buried in users' sequential behaviors. Despite the considerable achievements, existing methods usually focus on intra-sequence modeling while overlooking exploiting global collaborative information by inter-sequence modeling, resulting in inferior recommendation performance. Therefore, previous works attempt to tackle this problem with a global collaborative item graph constructed by pre-defined rules. However, these methods neglect two crucial properties when capturing global collaborative information, i.e., adaptiveness and personalization, yielding sub-optimal user representations. To this end, we propose a graph-driven framework, named Adaptive and Personalized Graph Learning for Sequential Recommendation (APGL4SR), that incorporates adaptive and personalized global collaborative information into sequential recommendation systems. Specifically, we first learn an adaptive global graph among all items and capture global collaborative information with it in a self-supervised fashion, whose computational burden can be further alleviated by the proposed SVD-based accelerator. Furthermore, based on the graph, we propose to extract and utilize personalized item correlations in the form of relative positional encoding, which is a highly compatible manner of personalizing the utilization of global collaborative information. Finally, the entire framework is optimized in a multi-task learning paradigm, thus each part of APGL4SR can be mutually reinforced. As a generic framework, APGL4SR can not only outperform other baselines with significant margins, but also exhibit promising versatility, the ability to learn a meaningful global collaborative graph, and the ability to alleviate the dimensional collapse issue of item embeddings.