Balancing the management of technical debt within recommender systems requires effectively juggling the introduction of new features with the ongoing maintenance and enhancement of the current system. Within the realm of recommender systems, technical debt encompasses the trade-offs and expedient choices made during the development and upkeep of the recommendation system, which could potentially have adverse effects on its long-term performance, scalability, and maintainability. In this vision paper, our objective is to kickstart a research direction regarding Technical Debt in AI-based Recommender Systems. We identified 15 potential factors, along with detailed explanations outlining why it is advisable to consider them.

The first objective of recommender systems is to provide personalized recommendations for each user. However, personalization may not be its only use. Past recommendations can be further analyzed to gain global insights into users’ behavior with respect to recommended items. Such insights can help to answer design-related questions such as which items’ recommendations are the most impactful in terms of users’ utility, which type of recommendations are the most followed ones, which items could be dropped from the catalog, or which recommendations are under-performing compared to what one would expect. In order to answer those questions, we need to rank item recommendations’ performances in terms of their causal impact on some user-related outcome measures. Unfortunately, in previous work leveraging causal inference for recommendation systems, the attention is fully focused on correcting confounding bias and not on the collider bias. This bias is particularly relevant in the recommender context, where multiple items are simultaneously recommended. Indeed, when there is a fixed number of available spots (i.e., k -spots) and recommendations need to be provided at each session, we argue that it is not possible to estimate the causal impacts of recommendations but only the differences between them. Therefore, in this article, we provide an unbiased estimator of the differences in the impacts of items’ recommendations, that work for any outcome of interest, and any type of recommender system as long as it has some degree of randomization. We apply our results both in a simulated environment and in a real-world offline environment leveraging logged data for recommended items in a digital healthcare app.

This paper presents an efficient algorithm to solve the sleeping bandit with multiple plays problem in the context of an online recommendation system. The problem involves bounded, adversarial loss and unknown i.i.d. distributions for arm availability. The proposed algorithm extends the sleeping bandit algorithm for single arm selection and is guaranteed to achieve theoretical performance with regret upper bounded by , where k is the number of arms selected per time step, N is the total number of arms, and T is the time horizon.

As stated in the United Nations’ Sustainable Development Goals, poor mental well-being is one of the biggest problems we are facing worldwide. One possible way of addressing it is through interventions delivered via digital devices since they are scalable, ubiquitous and inexpensive. This is also confirmed by the ever-growing plethora of e-health mobile apps being developed. Although these apps rely to some extent on scientific bases, there is still much work to do to understand the effect of specific digital interventions on app users. To shed light on these effects, we ask what types of interventions within the app have the most significant impact on well-being, and to what extent longer engagement leads to improved outcomes. These questions could be answered with dedicated Randomized Controlled Trials (RCTs), which are generally expensive, time-consuming, and single-purposed. To overcome these difficulties, we adopt instrumental variables on a combination of data collected in an RCT, behavioural data from the app, and a randomized recommender system, to evaluate intervention and app dose-response effects on users’ self-reported well-being. Thus, we present a general causal inference approach for extending results from collected data in RCTs applied in the context of digital health intervention. Following this approach, we show how to measure the impact of different types of activities on the users’ well-being. This allows us to identify the most impactful activities in the app (namely, sleep and relaxation activities), which have direct implications for the app design. On the other hand, we prove the positive effect of longer app usage.

Over the years, the Web has become a premier source of information in almost every area we can think about. When considering tourism, the Web became the primary source of information for travelers. When planning trips, people search for information about destinations, accommodations, attractions, means of transportation, in short, everything related to their future trip. Once done searching they reserve almost everything online. The blessing of the easily accessible information comes with the curse of information overload. This brings Web search techniques and recommendation systems come into play. This is especially true recently with the appearance of COVID-19 and the uncertainty and transformative power it brings to travelling. WebTour 2021 brings together researchers and practitioners working on developing and improving tools and techniques for improving users ability to better find relevant information that matches their needs.

The task “recommend a video to watch next?” has been in the focus of recommender systems’ research for a long time. However, adequately exploiting the clues hidden in the sequences of actions of user sessions in order to reveal users’ short-term intentions moved only recently into the focus of research. Based on a real-world application scenario, in this paper, we propose a Markov Chain-based transition probability matrix to efficiently reveal the short-term preferences of individuals. We experimentally evaluated our proposed method by comparing it against state-of-the-art algorithms in an offline as well as a live evaluation setting. In both cases our method not only demonstrated its superiority over its competitors, but exposed a clearly stronger engagement of users on the platform. In the online setting, our method improved the click-through rate by up to 93.61%. This paper therefore contributes real-world evidence for improving the recommendation effectiveness, by considering sequence-awareness, since capturing the short-term preferences of users is crucial in the light of items with a short life span such as tv programs (news, tv shows, etc.).

Rating-based summary statistics are ubiquitous in e-commerce, and often are crucial components in personalized recommendation mechanisms. Especially visual rating summarizations have been identified as important means to explain, why an item is presented or proposed to an user. Largely left unexplored, however, is the issue to what extent the descriptives of these rating summary statistics influence decision making of the online consumer. Therefore, we conducted a series of two conjoint experiments to explore how different summarizations of rating distributions (i.e., in the form of number of ratings, mean, variance, skewness, bimodality, or origin of the ratings) impact users' decision making. In a first study with over 200 participants, we identified that users are primarily guided by the mean and the number of ratings, and - to lesser degree - by the variance and origin of a rating. When probing the maximizing behavioral tendencies of our participants, other sensitivities regarding the summary of rating distributions became apparent. We thus instrumented a follow-up eye-tracking study to explore in more detail, how the choices of participants vary in terms of their decision making strategies. This second round with over 40 additional participants supported our hypothesis that users, who usually experience higher decision difficulty, follow compensatory decision strategies, and focus more on the decisions they make. We conclude by outlining how the results of these studies can guide algorithm development, and counterbalance presumable biases in implicit user feedback.

Collaborative filtering systems heavily depend on user feedback expressed in product ratings to select and rank items to recommend. These summary statistics of rating values carry two important descriptors about the assessed items, namely the total number of ratings and the mean rating value. In this study we explore how these two signals influence the decisions of online users based on choice-based conjoint experiments. Results show that users are more inclined to follow the mean indicator as opposed to the total number of ratings. Empirical results can serve as an input to developing algorithms that foster items with a, consequently, higher probability of choice based on their rating summarizations or their it explainability due to these ratings when ranking recommendations.

rrecsys is a novel library in R for developing and assessing recommendation algorithms. In this demo, we extend rrecsys with functions for visual analytics of recommendation performance, that is one of the strong capabilities of the R environment. In particular, we show how the library can be used to depict dataset characteristics, train and test recommendation algorithms and to visually assess, for instance, their capability to exploit long-tail items for making correct predictions.