With the growing influence of Recommender System (RS) in people’s daily lives, the issue of fairness in recommendations has become increasingly crucial. Previous fairness-aware methods focus on static or one-shot recommendation settings, where the recommendation model provides static fairness solutions via solving a fixed fairness-constrained optimization. However, these approaches face challenges in maintaining the delicate balance between recommendation accuracy and fairness in dynamic environments, as they fail to account for the evolving nature of RS, including changes in user preferences and item popularity over time.In this paper, we explore the problem of long-term fairness in interactive recommendation and accomplish the problem through dynamic fairness-constrained decisions. We focus on maintaining the exposure fairness of items across different groups. We argue that fairness does not always in conflict with recommendation performance, especially when considering the spatiotemporal heterogeneity of user preference on item popularity. To achieve this, we propose HER4IF, a dynamic fairness-aware interactive recommendation method based on a hierarchical reinforcement learning framework. Its main idea is first to aggregate the interacted item popularity with the time-forgetting model in state representation to capture user popularity preference. It then introduces a high-level agent to generate a dynamic fairness constraint based on the user’s current state, while a low-level agent generates recommendations under this constraint. Experiments on two datasets and an authentic Reinforcement Learning environment (KuaiSim) show the effectiveness and superiority of the proposed framework in terms of recommendation accuracy and fairness. It demonstrates a win-win for fairness and accuracy in a dynamic recommendation setting, when considering the dynamic nature of RS and incorporating spatiotemporal heterogeneity in user preference on item popularity.

While personalization increases the utility of item recommendation, it also suffers from the issue of popularity bias. However, previous methods emphasize adopting supervised learning models to relieve popularity bias in the static recommendation, ignoring the dynamic transfer of user preference and amplification effects of the feedback loop in the recommender system (RS). In this paper, we focus on studying this issue in the interactive recommendation. We argue that diversification and novelty are both equally crucial for improving user satisfaction of IRS in the aforementioned setting. To achieve this goal, we propose a D iversity- N ovelty- a ware I nteractive R ecommendation framework (DNaIR) that augments offline reinforcement learning (RL) to increase the exposure rate of long-tail items with high quality. Its main idea is first to aggregate the item similarity, popularity, and quality into the reward model to help the planning of RL policy. It then designs a diversity-aware stochastic action generator to achieve an efficient and lightweight DNaIR algorithm. Extensive experiments are conducted on the three real-world datasets and an authentic RL environment (Virtual-Taobao). The experiments show that our model can better and full use of the long-tail items to improve recommendation satisfaction, especially those low popularity items with high-quality ones, thus achieving state-of-the-art performance.

This article considers the problem of maintaining the long-term fairness of item exposure in interactive recommender systems under the dynamic setting that user preference and item popularity evolve over time. The challenge is that the evolving dynamics of user preference and item popularity in the feedback loop amplify the long-term “unfairness” of item exposure. To address this challenge, we first formulate a constrained Markov Decision Process (MDP) to capture the evolving dynamics of user preference. The proposed constrained MDP imposes long-term fairness requirements via maximum entropy techniques. Moreover, to illuminate the “unfairness” amplifying effect caused by the evolving dynamic of item popularity in the feedback loop, we design a debiased reward function to eliminate popularity bias in the training data. To this end, the proposed framework can maintain acceptable recommendation accuracy while exposing items as randomly as possible, ensuring long-term benefits for users. To address the data sparsity issue, the proposed framework can easily integrate self-supervised learning methods to enhance state representation. Experiments on three datasets and an authentic Reinforcement Learning environment (Virtual-Taobao) demonstrate the effectiveness and superiority of the proposed framework in terms of recommendation accuracy and fairness, and show the robustness against data sparsity and noise.

This paper considers the problem of maintaining the long-term fairness of item exposure in interactive recommendation systems under the dynamic setting that user preference and item popularity evolve over time. The challenge is that the evolving dynamics of user preference and item popularity in the feedback loop amplify the long-term “unfairness” of item exposure. To address this challenge, we first formulate a constrained Markov Decision Process (MDP) to capture evolving dynamics of user preference. The proposed constrained MDP imposes long-term fairness requirements via maximum entropy techniques. Moreover, to illuminate the “unfairness” amplifying effect caused by the evolving dynamic of item popularity in the feedback loop, we design a debiased reward function to eliminate popularity bias in the training data. To this end, the proposed framework can maintain acceptable recommendation accuracy while exposing items as randomly as possible, ensuring long-term benefits for users. Experiments on three datasets demonstrate the effectiveness and superiority of our proposed framework in terms of recommendation performance and fairness.