Interactive Recommendation (IR) has gained significant attention recently for its capability to quickly capture dynamic interest and optimize both short and long term objectives. IR agents are typically implemented through Deep Reinforcement Learning (DRL), because DRL is inherently compatible with the dynamic nature of IR. However, DRL is currently not perfect for IR. Due to the large action space and sample inefficiency problem, training DRL recommender agents is challenging. The key point is that useful features cannot be extracted as high-quality representations for the recommender agent to optimize its policy. To tackle this problem, we propose Contrastive Representation for Interactive Recommendation (CRIR). CRIR efficiently extracts latent, high-level preference ranking features from explicit interaction, and leverages the features to enhance users’ representation. Specifically, the CRIR provides representation through one representation network, and refines it through our proposed Preference Ranking Contrastive Learning (PRCL). The key insight of PRCL is that it can perform contrastive learning without relying on computations involving high-level representations or large potential action sets. Furthermore, we also propose a data exploiting mechanism and an agent training mechanism to better adapt CRIR to the DRL backbone. Extensive experiments have been carried out to show our method's superior improvement on the sample efficiency while training an DRL-based IR agent.

Efficient exploration in high-dimensional observation spaces remains a critical challenge in deep reinforcement learning, particularly in scenarios with sparse extrinsic rewards. A promising approach is to encourage exploration by estimating intrinsic rewards based on the novelty of observations. However, there is a gap between the observed novelty and the actual effectiveness of exploration, as both environmental stochasticity and the agent’s actions may influence observations. To accurately evaluate the novelty contributed by agent exploration in intrinsic rewards, we propose the AIRES (Attention-driven Intrinsic Reward for Exploration Strategy) framework. AIRES leverages the attention mechanisms to analyze the relationship within trajectory sequences generated by agent-environment interactions, employing attention weights to quantify the relevance of observations to actions. By applying attention weights to intrinsic rewards, the novelty brought by agent exploration is enhanced and the impact of environmental stochasticity is reduced. Extensive experiments demonstrate that AIRES significantly enhances the performance of prominent intrinsic reward methods, establishing it as a robust and scalable solution for efficient exploration.

Traditional recommendation systems focus on maximizing user satisfaction by suggesting their favorite items. This user-centric approach may lead to unfair exposure distribution among the providers. On the contrary, a provider-centric design might become unfair to the users. Therefore, this paper proposes a re-ranking model FairSort1to find a trade-off solution among user-side fairness, provider-side fairness, and personalized recommendations utility. Previous works habitually treat this issue as a knapsack problem, incorporating both-side fairness as constraints. In this paper, we adopt a novel perspective, treating each recommendation list as a runway rather than a knapsack. In this perspective, each item on the runway gains a velocity and runs within a specific time, achieving re-ranking for both-side fairness. Meanwhile, we ensure the Minimum Utility Guarantee for personalized recommendations by designing a Binary Search approach. This can provide more reliable recommendations compared to the conventional greedy strategy based on the knapsack problem. We further broaden the applicability of FairSort, designing two versions for online and offline recommendation scenarios. Theoretical analysis and extensive experiments on real-world datasets indicate that FairSort can ensure more reliable personalized recommendations while considering fairness for both the provider and user.

Figure question answering (FQA) is a new multimodal task for visual question answering (VQA). Given a scientific-style Figure and a related question, the machine must answer the question through reasoning. The Relation Networks (RN), the earliest proposed approach for FQA, computes the representation of relations between objects within images to infer the answers. However, RN generates numerous relation features, which makes the reasoning process more complicated and restricts the performance. To better solve this problem, we introduce a novel framework, which consists of a deconvolutional network, an LSTM network and an affinitydriven relation network. Specifically, the deconvolutional network enhances the feature fusion by combining low-level and high-level features of images. The affinity-driven relation network efficiently represents the intra-relation within images and the inter-relation between images and questions, and makes the reasoning process more effective. The experimental results show that our approach outperforms most state-of the-art methods in FQA tasks.