Recommender system helps address information overload problem and satisfy consumers’ personalized requirement in many applications such as e-commerce, social networks, and in-game store. However, existing approaches mainly focus on improving the accuracy of recommendation tasks but usually ignore how to improve the interpretability of recommendation, which is still a challenging and crucial task, especially for some complicated scenarios such as large-scale online games. A few previous attempts on explainable recommendation mostly depend on a large amount of a priori knowledge or user-provided review corpus, which is labor consuming as well as often suffers from data deficiency. To relieve this issue, we propose a Multi-source Heterogeneous Graph Attention Network for Explainable Recommendation (MHANER) for the case without enough a priori knowledge or corpus of user comments. Specifically, MHANER employs the attention mechanism to model players’ preference to in-game store items as the support for the explanation of recommendation. Then a graph neural network–based method is designed to model players’ multi-source heterogeneous information, including the players’ historical behavior data, historical purchase data, and attributes of the player-controlled character, which is leveraged to recommend possible items for players to buy. Finally, the multi-level subgraph pattern mining is adopted to combine the characteristics of a recommendation list to generate corresponding explanations of items. Extensive experiments on three real-world datasets, two collected from JD and one from NetEase game, demonstrate that the proposed model MHANER outperforms state-of-the-art baselines. Moreover, the generated explanations are verified by human encoding comprised of hard-core game players and endorsed by experts from game developers.

Reinforcement learning based recommender systems (RL-based RS) aim at learning a good policy from a batch of collected data, by casting recommendations to multi-step decision-making tasks. However, current RL-based RS research commonly has a large reality gap. In this paper, we introduce the first open-source real-world dataset, RL4RS, hoping to replace the artificial datasets and semi-simulated RS datasets previous studies used due to the resource limitation of the RL-based RS domain. Unlike academic RL research, RL-based RS suffers from the difficulties of being well-validated before deployment. We attempt to propose a new systematic evaluation framework, including evaluation of environment simulation, evaluation on environments, and counterfactual policy evaluation. In summary, the RL4RS (Reinforcement Learning for Recommender Systems), a new resource with special concerns on the reality gaps, contains two real-world datasets, data understanding tools, tuned simulation environments, related advanced RL baselines, batch RL baselines, and counterfactual policy evaluation algorithms. The RL4RS suite can be found at https://github.com/fuxiAIlab/RL4RS.

In recent years, there are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems (RS). In this paper, we summarize three key practical challenges of large-scale RL-based recommender systems: massive state and action spaces, high-variance environment, and the unspecific reward setting in recommendation. All these problems remain largely unexplored in the existing literature and make the application of RL challenging. We develop a model-based reinforcement learning framework, called GoalRec. Inspired by the ideas of world model (model-based), value function estimation (model-free), and goal-based RL, a novel disentangled universal value function designed for item recommendation is proposed. It can generalize to various goals that the recommender may have, and disentangle the stochastic environmental dynamics and high-variance reward signals accordingly. As a part of the value function, free from the sparse and high-variance reward signals, a high-capacity reward-independent world model is trained to simulate complex environmental dynamics under a certain goal. Based on the predicted environmental dynamics, the disentangled universal value function is related to the user's future trajectory instead of a monolithic state and a scalar reward. We demonstrate the superiority of GoalRec over previous approaches in terms of the above three practical challenges in a series of simulations and a real application.

In the business domain,bundling is one of the most important marketing strategies to conduct product promotions, which is commonly used in online e-commerce and offline retailers. Existing recommender systems mostly focus on recommending individual items that users may be interested in, such as the considerable research work on collaborative filtering that directly models the interaction between users and items. In this paper, we target at a practical but less explored recommendation problem named personalized bundle composition, which aims to offer an optimal bundle (i.e., a combination of items) to the target user. To tackle this specific recommendation problem, we formalize it as a combinatorial optimization problem on a set of candidate items and solve it within a neural combinatorial optimization framework. Extensive experiments on public datasets are conducted to demonstrate the superiority of the proposed method.

In business domains, bundling is one of the most important marketing strategies to conduct product promotions, which is commonly used in online e-commerce and offline retailers. Existing recommender systems mostly focus on recommending individual items that users may be interested in. In this paper, we target at a practical but less explored recommendation problem named bundle recommendation, which aims to offer a combination of items to users. To tackle this specific recommendation problem in the context of the virtual mall in online games, we formalize it as a link prediction problem on a user-item-bundle tripartite graph constructed from the historical interactions, and solve it with a neural network model that can learn directly on the graph-structure data. Extensive experiments on three public datasets and one industrial game dataset demonstrate the effectiveness of the proposed method. Further, the bundle recommendation model has been deployed in production for more than one year in a popular online game developed by Netease Games, and the launch of the model yields more than 60% improvement on conversion rate of bundles, and a relative improvement of more than 15% on gross merchandise volume (GMV).