Frequent updates and model retraining are important in various application areas of recommender systems, e.g., news recommendation. Moreover, in such domains, we may not only face the problem of dealing with a constant stream of new data, but also with anonymous users, leading to the problem of streaming session-based recommendation (SSR). Such problem settings have attracted increased interest in recent years, and different deep learning architectures were proposed that support fast updates of the underlying prediction models when new data arrive. In a recent paper, a method based on Graph Neural Networks (GNN) was proposed as being superior than previous methods for the SSR problem. The baselines in the reported experiments included different machine learning models. However, several earlier studies have shown that often conceptually simpler methods, e.g., based on nearest neighbors, can be highly effective for session-based recommendation problems. In this work, we report a similar phenomenon for the streaming configuration. We first reproduce the results of the mentioned GNN method and then show that simpler methods are able to outperform this complex state-of-the-art neural method on two datasets. Overall, our work points to continued methodological issues in the academic community, e.g., in terms of the choice of baselines and reproducibility.1

Sequential recommendation problems have received increased research interest in recent years. In such scenarios, the task is to suggest items to users to consume next, given their past interaction history, e.g., the next movie to watch or the next item to place in the shopping cart. A number of machine learning models were proposed recently for the task of sequential recommendation, with the latest ones based on deep learning techniques, in particular on Transformers. Given the often surprisingly competitive performance of simpler nearest-neighbor methods for the related problem of session-based recommendation, we investigate the use of nearest-neighbor methods for sequential recommendation problems. Our analysis on four datasets shows that nearest-neighbor methods achieve comparable or better performance than the recent Transformer-based bert4rec method on two of them. However, the deep learning method outperforms the simple methods for the two larger datasets, confirming previous hypotheses that neural methods work best when more data is available. As a further result of our experiments, we found additional evidence that sampled metrics must be used with care, as they may not be predictive of an algorithm ranking that would be observed with the non-sampled, full evaluation.

Recommender systems are designed to help users in situations of information overload. In recent years we observed increased interest in session-based recommendation scenarios, where the problem is to make item suggestions to users based only on interactions observed in an ongoing session, e.g., on an e-commerce site. However, in cases where interactions from previous user sessions are also available, the recommendations can be personalized according to the users’ long-term preferences, a process called session-aware recommendation. Today, research in this area is scattered, and many works only compare a newly proposed session-aware with existing session-based models. This makes it challenging to understand what represents the state-of-the-art. To close this research gap, we benchmarked recent session-aware algorithms against each other and against a number of session-based recommendation algorithms along with heuristic extensions thereof. Our comparison, to some surprise, revealed that (i) simple techniques based on nearest neighbors consistently outperform recent neural techniques and that (ii) session-aware models were mostly not better than approaches that do not use long-term preference information. Our work therefore points to potential methodological issues where new methods are compared to weak baselines, and it also indicates that there remains a huge potential for more sophisticated session-aware recommendation algorithms.

Abstract Recommender systems are tools that support online users by pointing them to potential items of interest in situations of information overload. In recent years, the class of session-based recommendation algorithms received more attention in the research literature. These algorithms base their recommendations solely on the observed interactions with the user in an ongoing session and do not require the existence of long-term preference profiles. Most recently, a number of deep learning-based (“neural”) approaches to session-based recommendations have been proposed. However, previous research indicates that today’s complex neural recommendation methods are not always better than comparably simple algorithms in terms of prediction accuracy. With this work, our goal is to shed light on the state of the art in the area of session-based recommendation and on the progress that is made with neural approaches. For this purpose, we compare twelve algorithmic approaches, among them six recent neural methods, under identical conditions on various datasets. We find that the progress in terms of prediction accuracy that is achieved with neural methods is still limited. In most cases, our experiments show that simple heuristic methods based on nearest-neighbors schemes are preferable over conceptually and computationally more complex methods. Observations from a user study furthermore indicate that recommendations based on heuristic methods were also well accepted by the study participants. To support future progress and reproducibility in this area, we publicly share the session-rec evaluation framework that was used in our research.

The benefits of neural approaches are undisputed in many application areas. However, today's research practice in applied machine learning---where researchers often use a variety of baselines, datasets, and evaluation procedures---can make it difficult to understand how much progress is actually achieved through novel technical approaches. In this work, we focus on the fast-developing area of session-based recommendation and aim to contribute to a better understanding of what represents the state-of-the-art.