Reinforcement learning (RL) has demonstrated great potential to improve slate-based recommender systems by optimizing long-term user engagement. However, addressing the combinatorial action space in slate recommendations remains challenging. Recent work decomposes slate Q -values into item-wise Q -values, improving the tractability of value-based methods to learn the model. But in scenarios with a large item pool and a resource-intensive value function like deep neural networks, the action selection process still incurs substantial computational costs. Slow training might be tolerable, but high costs during action selection could hinder real-time deployment. To address this issue, this article introduces an actor method that reduces Q -function evaluations to a subset of items, significantly cutting inference time for practical deployment. The research suggests acquiring representations at both item and slate levels, strategically identifying a specific item subset for slate composition. The proposed methodologies are assessed over different simulated user engagement behaviors: users certain about preferences (“decisive” behavior) and those more exploratory or bored users, losing interest with repetitive content exposure (“explorative” behavior). Empirical evaluation shows that the proposed approach achieves comparable user engagement with a value-based policy across behaviors. Meanwhile, it notably enhances serving time while recommending diverse topic slates, thus demonstrating its potential effectiveness and efficiency in real-world applications.

Session-based slate recommendation systems curate ranked sets of items in real-time, adapting to evolving user interactions.Balancing relevance, diversity, and novelty remains challenging for reinforcement learning (RL) methods.Recent advances in large language models (LLMs) offer a new possibility to leverage their semantic reasoning capabilities to refine slate composition.In this work, we examine the impact of LLM-driven reasoning on slate generation by integrating LLMs with an RL-based slate recommender and evaluating in terms of accuracy, similarity, diversity, and novelty.We extend the RecSim framework with real-world interaction data and introduce a session-aware evaluation protocol that captures long-term engagement.Our analysis reveals that LLM reasoning enhances subcategory-level diversity while maintaining relevance, leading to increased user engagement.By visualizing category-level shifts in slate composition we uncover systematic patterns in how LLMs refine recommendation diversity.Although direct LLM use during inference may be hampered by computational demands and latency concerns, our experimental results demonstrate that integrating LLM modifications during training enables the model to internalize the nuanced characteristics of LLM reasoning without incurring inference overhead, thereby improving recommendation performance, serving time efficiency, and deployability.

Reinforcement learning (RL) has demonstrated great potential to improve slate-based recommender systems by optimizing recommendations for long-term user engagement. To handle the combinatorial action space in slate recommendation, recent works decompose the Q-value of a slate into item-wise Q-values, using an item-wise value-based policy. However, the common case where the value function is a parameterized function taking state and action as input results in a linearly increasing number of evaluations required to select an action, proportional to the number of candidate items. While slow training may be acceptable, this becomes intractable when considering the costly evaluation of the parameterized function, such as with deep neural networks, during model serving time. To address this issue, we propose an actor-based policy that reduces the evaluation of the Q-function to a subset of items, significantly reducing inference time and enabling practical deployment in real-world industrial settings. In our empirical evaluation, we demonstrate that our proposed approach achieves equivalent user session engagement to a value-based policy, while significantly reducing the slate serving time by at least 4 times.