Conventional multi-modal sequential recommenders usually employ sequence models (e.g., SASRec and GRU4Rec) as the recommender framework for sequential dependency modeling and multi-modal information as add-on knowledge for fine-grained preference learning. However, the existing methods ignore the time interval information in the interaction sequences, which reflects the user’s preference transition. Consequently, they acquire biased preference representations, leading to suboptimal performance. Along these lines, we concentrate on incorporating the time interval into the multi-modal sequential recommendation and investigating the internal influences across multiple modalities. Firstly, we treat the time interval as an independent modality and exploit a Time Interval Encoder (TIE) to quantify the time interval in the sequences. Secondly, we consider the heterogeneity between multiple modalities and design a two-stage fusion strategy. The first stage injects time intervals into each modality (ID, image, and text) and the second stage aggregates the user preferences from each modality to get the user’s sequence preferences. To this end, we build a model-agnostic framework for the multi-modal sequential recommendation, namely the Time-interval-aware Multi-Modal Sequential Recommender (TMMSRec) . The empirical validations on four real-world datasets prove the effectiveness of our method, where the maximum performance improvement against several state-of-the-art baselines achieves up to 16.05%. Our code is available at https://github.com/wangpp0602/TMMSRec .

Sequential models aim to predict future interactions based on users' historical interaction sequences. Traditional sequential methods primarily focus on capturing intra-historical sequence dependencies, overlooking the influence of unobserved confounders in recommendation scenarios. Recent studies incorporate time as additional information helps the model capture dynamic user preferences. However, time is just the external manifestation of the influence of confounders but not the actual cause of the dynamic of user preference. Additionally, improperly integrating time with item embeddings can obstruct the model's ability to capture sequence dependencies. To address these challenges, we first revisit the sequential recommendation problem from a causal perspective and incorporate confounders as a new task. We propose a new framework—Flow-based Time-aware Causal Structure for Sequential Recommendation (FCSRec)—explicitly incorporating unobserved confounders' influence in the recommendation process. Specifically, we use Normalizing Flows to learn the causal graph of confounders and incorporate time information as conditional info to capture confounders' time-sensitive representations. To balance the influence of confounders and sequence dependencies, we introduce a classifier-free training paradigm by randomly masking the influence of confounders during training to encourage the model to learn both sequence dependencies and confounders' influence equally. We validate FCSRec on manifold real-world datasets, and experimental results show that FCSRec outperforms several state-of-the-art methods in recommendation performance. Our code is available at Code-link.

Inferring user preferences from users’ historical feedback is a valuable problem in recommender systems. Conventional approaches often rely on the assumption that user preferences in the feedback data are equivalent to the real user preferences without additional noise, which simplifies the problem modeling. However, there are various confounders during user–item interactions, such as weather and even the recommendation system itself. Therefore, neglecting the influence of confounders will result in inaccurate user preferences and suboptimal performance of the model. Furthermore, the unobservability of confounders poses a challenge in further addressing the problem. Along these lines, we refine the problem and propose a more rational solution to mitigate the influence of unobserved confounders. Specifically, we consider the influence of unobserved confounders, disentangle them from user preferences in the latent space, and employ causal graphs to model their interdependencies without specific labels. By ingeniously combining local and global causal graphs, we capture the user-specific effects of confounders on user preferences. Finally, we propose our model based on Variational Autoencoders, named Causal Structure Aware Variational Autoencoders (CSA-VAE) and theoretically demonstrate the identifiability of the obtained causal graph. We conducted extensive experiments on one synthetic dataset and nine real-world datasets with different scales, including three unbiased datasets and six normal datasets, where the average performance boost against several state-of-the-art baselines achieves up to 9.55%, demonstrating the superiority of our model. Furthermore, users can control their recommendation list by manipulating the learned causal representations of confounders, generating potentially more diverse recommendation results. Our code is available at Code-link ( https://github.com/MICLab-Rec/CSA ).