The 12th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems (IntRS'25) takes a user-centric perspective on recommender systems research. It brings together an interdisciplinary community of researchers and practitioners who explore a range of important issues on the "user side"of recommender systems, including user studies, psychology-informed design of RSs, novel interfaces, and evaluation methodologies. Its purpose is to identify critical challenges and emerging topics with a strong focus on fundamental historical challenges in the field. In this summary, we introduce the motivation and perspective of the workshop, review its history, and discuss the most critical issues that deserve attention for future research directions.

The primary goal of Recommender Systems is to suggest the most suitable items to a user, aligning them with the user’s interests and needs. RSs are essential for modern e-commerce, helping users discover content and products by predicting suitable items based on their past behavior. However, their success isn’t just about advanced algorithms. The design of the user interface and a good integration with the human decision-making process are equally crucial. A well-designed interface enhances the user experience and makes recommendations more effective, while a poor interface can lead to frustration. Recognizing this limitation, recent trends in Recommender Systems (RSs) are increasingly focusing on integrating Symbiotic Human-Machine Decision-Making models. These models aim to offer users a dynamic and persuasive interface that helps them better understand and engage with recommendations. This shift is a crucial step toward developing recommender systems that truly connect with users and offer a more enjoyable, trustworthy, explainable, and user-friendly experience. Although early efforts concentrated on creating systems that could proactively predict user preferences and needs, modern RSs also emphasize the importance of providing users with control and transparency over their recommendations. Finding the right balance between proactivity and user control is essential to ensure that the system supports users without being too intrusive, thus improving their overall satisfaction. As Large Language Models (LLMs) become more integrated into recommender systems, the importance of user-centric interfaces and a deep understanding of decision-making becomes even more critical. Effective integration of LLMs requires interfaces that are both visually and cognitively engaging.

Recommender systems (RSs) have undoubtedly played a significant role in addressing the information overload problem by efficiently filtering and suggesting relevant items to users. These systems use both explicit and implicit user preferences to filter available data and suggest items that might align with the user’s interests. This can range from recommending movies on a streaming platform based on previous views to suggesting products for purchase based on browsing history. In their early stages, RSs focused on enhancing their algorithmic capabilities to provide accurate recommendations. However, the overemphasis on algorithms resulted in neglecting the human aspect of the user experience. Recognizing this limitation, recent trends in RSs have started to shift their attention toward incorporating Symbiotic Human-Machines Decision Making models. These models aim to provide users with dynamic and persuasive interfaces that empower them to understand and engage better with the recommendations. This shift represents an essential step in creating recommender systems that truly resonate with users and create a more enjoyable, trustable, and user-friendly experience. A crucial aspect of recommender systems’ evolution lies in their proactive nature. Early works focused on designing systems that could proactively anticipate user preferences and needs. While this remains a valuable trait, modern RSs also recognize the importance of giving users control and transparency over their recommendations. Striking the right balance between proactivity and user control ensures that the system supports users without being overly intrusive, thus enhancing their overall satisfaction. These aspects are the main discussion topics of the Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at RecSys’23. In this summary, we introduce the workshop’s motivation and view, review its history, and discuss the most critical issues that deserve attention for future research directions.

The constant increase in the amount of data and information available on the Web has made the development of systems that can support users in making relevant decisions increasingly important. Recommender systems (RSs) have emerged as tools to address this task. RSs use the preferences expressed by a user, either explicitly or implicitly, to filter the available information and proactively suggest items that might be of interest to him or her. Although in early works about the topic there was a strong interest in ways to make such systems proactive, user-friendly, and persuasive, over time they became increasingly focused on the algorithmic component solely. However, this trend is gradually being reversed and always more attention is nowadays placed also on Human Decision Making models that focus on supporting the end user in understanding what is being proposed through RSs by using dynamic and persuasive interfaces. A recommender system should be based on valuable strategies for proactively guiding users to items that match their preferences and therefore should put attention on how it is possible to make this process trustable, pleasant, and user-friendly. Such systems, moreover, should take into account psychological, cognitive and emotional aspects to enable personalization that is appropriate not only to the context of use but also to the psychological reactions of the end user. The workshop provides a venue for works that invest in the design of recommender systems which consider users’ experience during the interaction, as well as for works that explore the implications of human-computer interactions with different theories of human decision-making. In this summary, we introduce the Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at RecSys’22, review its history, and discuss the most important topics considered at the workshop.

Previous studies on exploration have shown that users can be nudged to explore further away from their current preferences. However, these effects were shown in a single session study, while it often takes time to explore new tastes and develop new preferences. In this work, we present a longitudinal study on users’ exploration behavior and behavior change over time after they have used a music genre exploration tool for four sessions in six weeks. We test two relevant nudges to help them explore more: the starting point (the personalization of the default initial playlist) and the visualization of users’ previous position(s). Our results show that the personalization level of the default initial playlist in the first session influences the preferred personalization level users set in the second session but fades away in later sessions as users start exploring in different directions. Visualization of users’ previous positions did not anchor users to stay closer to the initial defaults. Over time, users perceived the playlist to be more personalized to their tastes and helpful to explore the genre. Perceived helpfulness increased more when users explored further away from their current preferences. Apart from differences in self-reported measures, we also find some objective evidence for preference change in users’ top tracks from their Spotify profile, that over the period of 6 weeks moved somewhat closer to the genre that users selected to explore with the tool.

Recommender systems were originally developed as interactive intelligent systems that can proactively guide users to items that match their preferences. Despite its origin on the crossroads of HCI and AI, the majority of research on recommender systems gradually focused on objective accuracy criteria paying less and less attention to how users interact with the system as well as the efficacy of interface designs from users’ perspectives. This trend is reversing with the increased volume of research that looks beyond algorithms, into users’ interactions, decision making processes, and overall experience. The series of workshops on Interfaces and Human Decision Making for Recommender Systems focuses on the ”human side” of recommender systems. The goal of the research stream featured at the workshop is to improve users’ overall experience with recommender systems by integrating different theories of human decision making into the construction of recommender systems and exploring better interfaces for recommender systems. In this summary, we introduce the Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at RecSys’21, review its history, and discuss most important topics considered at the workshop.

Recommender systems are efficient at predicting users’ current preferences, but how users’ preferences develop over time is still under-explored. In this work, we study the development of users’ musical preferences. Exploring musical preference consistency between short-term and long-term preferences in data from earlier studies, we find that users with higher musical expertise have more consistent preferences at their top-listened artists and tags than those with lower musical expertise. Users typically chose to explore genres that were close to their current preferences, and this effect was stronger for expert users. Based on these findings we conducted a user study on genre exploration to investigate (1) whether it is possible to nudge users to explore more distant genres, and (2) how users’ exploration behaviors within a genre are influenced by default recommendation settings that balance personalization with genre representativeness in different ways. Our results show that users were more likely to select the more distant genres if these were presented at the top of the list. However, users with high musical expertise were less likely to do so, consistent with our earlier findings. When given a representative or mixed (balanced) default for exploration within a genre, users selected less personalized recommendation settings and explored further away from their current preferences, than with a personalized default. However, this effect was moderated by users’ slider usage behaviors. Overall, our results suggest that (personalized) defaults can nudge users to explore new, more distant genres and songs. However, the effect is smaller for those with higher musical expertise levels.

As an interactive intelligent system, recommender systems are developed to give recommendations that match users’ preferences. Since the emergence of recommender systems, a large majority of research focuses on objective accuracy criteria and less attention has been paid to how users interact with the system and the efficacy of interface designs from users’ perspectives. The field has reached a point where it is ready to look beyond algorithms, into users’ interactions, decision making processes, and overall experience. The series of workshops on Interfaces and Human Decision Making for Recommender Systems focuses on the ”human side” of recommender systems. The goal of the research stream featured at the workshop is to improve users’ overall experience with recommender systems by integrating different theories of human decision making into the construction of recommender systems and exploring better interfaces for recommender systems. In this summary, we introduce 7th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at RecSys’20, review its history, and discuss most important topics considered at the workshop.

As an interactive intelligent system, recommender systems are developed to give recommendations that match users' preferences. Since the emergence of recommender systems, a large majority of research focuses on objective accuracy criteria and less attention has been paid to how users interact with the system and the efficacy of interface designs from users' perspectives. The field has reached a point where it is ready to look beyond algorithms, into users' interactions, decision making processes, and overall experience. This workshop will focus on the "human side" of recommender systems research. The workshop goal is to improve users' overall experience with recommender systems by integrating different theories of human decision making into the construction of recommender systems and exploring better interfaces for recommender systems.

User-system interaction in recommender systems involves three aspects: temporal browsing (viewing recommendation lists and/or searching/filtering), action (performing actions on recommended items, e.g., clicking, consuming) and inaction (neglecting or skipping recommended items). Modern recommenders build machine learning models from recordings of such user interaction with the system, and in doing so they commonly make certain assumptions (e.g., pairwise preference orders, independent or competitive probabilistic choices, etc.). In this paper, we set out to study the effects of these assumptions along three dimensions in eight different single models and three associated hybrid models on a user browsing data set collected from a real-world recommender system application. We further design a novel model based on recurrent neural networks and multi-task learning, inspired by Decision Field Theory, a model of human decision making. We report on precision, recall, and MAP, finding that this new model outperforms the others.

As an interactive intelligent system, recommender systems are developed to give recommendations that match users' preferences. Since the emergence of recommender systems, a large majority of research focuses on objective accuracy criteria and less attention has been paid to how users interact with the system and the efficacy of interface designs from users' perspectives. The field has reached a point where it is ready to look beyond algorithms, into users' interactions, decision making processes, and overall experience. his workshop will focus on the "human side" of recommender systems research. The workshop goal is to improve users' overall experience with recommender systems by integrating different theories of human decision making into the construction of recommender systems and exploring better interfaces for recommender systems.

Temporally, users browse and interact with items in recommender systems. However, for most systems, the majority of the displayed items do not elicit any action from users. In other words, the user-system interaction process includes three aspects: browsing, action, and inaction. Prior recommender systems literature has focused more on actions than on browsing or inaction. In this work, we deployed a field survey in a live movie recommender system to interpret what inaction means from both the user's and the system's perspective, guided by psychological theories of human decision making. We further systematically study factors to infer the reasons of user inaction and demonstrate with offline data sets that this descriptive and predictive inaction model can provide benefits for recommender systems in terms of both action prediction and recommendation timing.

As intelligent interactive systems, recommender systems focus on determining predictions that fit the wishes and needs of users. Still, a large majority of recommender systems research focuses on accuracy criteria and much less attention is paid to how users interact with the system, and in which way the user interface has an influence on the selection behavior of the users. Consequently, it is important to look beyond algorithms. The main goals of the IntRS workshop are to analyze the impact of user interfaces and interaction design, and to explore human interaction with recommender systems from a human decision making perspective. Methodologies for evaluating these aspects are also within the scope of the workshop.

People often struggle to find appropriate energy-saving measures to take in the household. Although recommender studies show that tailoring a system's interaction method to the domain knowledge of the user can increase energy savings, they did not actually tailor the conservation advice itself. We present two large user studies in which we support users to make an energy-efficient behavioral change by presenting tailored energy-saving advice. Both systems use a one-dimensional, ordinal Rasch scale, which orders 79 energy-saving measures on their behavioral difficulty and link this to a user's energy-saving ability for tailored advice. We established that recommending Rasch-based advice can reduce a user's effort, increase system support and, in turn, increase choice satisfaction and lead to the adoption of more energy-saving measures. Moreover, follow-up surveys administered four weeks later point out that tailoring advice on its feasibility can support behavioral change.

As intelligent interactive systems, recommender systems focus on determining predictions that fit the wishes and needs of users. Still, a large majority of recommender systems research focuses on accuracy criteria and much less attention is paid to how users interact with the system, and in which way the user interface has an influence on the selection behavior of the users. Consequently, it is important to look beyond algorithms. The main goals of the IntRS workshop are to analyze the impact of user interfaces and interaction design, and to explore human interaction with recommender systems. Methodologies for evaluating these aspects are also within the scope of the workshop.

Behaviorism is the currently-dominant paradigm for building and evaluating recommender systems. Both the operation and the evaluation of recommender system applications are most often driven by analyzing the behavior of users. In this paper, we argue that listening to what users say about the items and recommendations they like, the control they wish to exert on the output, and the ways in which they perceive the system and not just observing what they do will enable important developments in the future of recommender systems. We provide both philosophical and pragmatic motivations for this idea, describe the various points in the recommendation and evaluation processes where explicit user input may be considered, and discuss benefits that may result from considered incorporation of user preferences at each of these points. In particular, we envision recommender applications that aim to support users' better selves: helping them live the life that they desire to lead. For example, recommender-assisted behavior change requires algorithms to predict not what users choose or do now, inferable from behavioral data, but what they should choose or do in the future to become healthier, fitter, more sustainable, or culturally aware. We hope that our work will spur useful discussion and many new ideas for recommenders that empower their users.

We studied an alternative choice-based interface for preference elicitation during the cold start phase and compared it directly with a standard rating-based interface. In this alternative interface users started from a diverse set covering all movies and iteratively narrowed down through a matrix factorization latent feature space to smaller sets of items based on their choices. The results show that compared to a rating-based interface, the choice-based interface requires less effort and results in more satisfying recommendations, showing that it might be a promising candidate for alleviating the cold start problem of new users.

Recent developments in user evaluation of recommender systems have brought forth powerful new tools for understanding what makes recommendations effective and useful. We apply these methods to understand how users evaluate recommendation lists for the purpose of selecting an algorithm for finding movies. This paper reports on an experiment in which we asked users to compare lists produced by three common collaborative filtering algorithms on the dimensions of novelty, diversity, accuracy, satisfaction, and degree of personalization, and to select a recommender that they would like to use in the future. We find that satisfaction is negatively dependent on novelty and positively dependent on diversity in this setting, and that satisfaction predicts the user's final selection. We also compare users' subjective perceptions of recommendation properties with objective measures of those same characteristics. To our knowledge, this is the first study that applies modern survey design and analysis techniques to a within-subjects, direct comparison study of recommender algorithms.

A primary function of recommender systems is to help their users to make better choices and decisions. The overall goal of the workshop is to analyse and discuss novel techniques and approaches for supporting effective and efficient human decision making in different types of recommendation scenarios. The submitted papers discuss a wide range of topics from core algorithmic issues to the management of the human computer interaction.

One of the challenges for recommender systems is that users struggle to accurately map their internal preferences to external measures of quality such as ratings. We study two methods for supporting the mapping process: (i) reminding the user of characteristics of items by providing personalized tags and (ii) relating rating decisions to prior rating decisions using exemplars. In our study, we introduce interfaces that provide these methods of support. We also present a set of methodologies to evaluate the efficacy of the new interfaces via a user experiment. Our results suggest that presenting exemplars during the rating process helps users rate more consistently, and increases the quality of the data.

Many recommendation systems rely on explicit ratings provided by their users. Often these ratings are provided long after consuming the item, relying heavily on people's representation of the quality of the item in memory. This paper investigates a psychological process, the "positivity effect", that influences the retrieval of quality judgments from our memory by which pleasant items are being processed and recalled from memory more effectively than unpleasant items. In an offline study on the MovieLens data we used the time between release date and rating date as a proxy for the time between consumption and rating. Ratings for movies tend to increase over time, consistent with the positivity effect. A subsequent online user study used a direct measure of time between rating and consumption, by asking users to rate movies (recently aired on television) and to explicitly report how long ago they watched these movies. In contrast to the offline study we find that ratings tend to decline over time showing reduced accuracy in ratings for items experienced long ago. We discuss the impact these rating dynamics might have on recommender algorithms, especially in cases where a new user has to submit his preferences to a system.

Interacting with a recommender system means to take different decisions such as selecting an item from a recommendation list, selecting a specific item feature value (e.g., camera's size, zoom) as a search criteria, selecting feedback features to be critiqued in a critiquing based recommendation session, or selecting a repair proposal for inconsistent user preferences when interacting with a knowledge-based recommender. In all these situations, users face a decision task. This workshop ([email protected]) focuses on approaches for supporting effective and efficient human decision making in different types of recommendation scenarios.

This paper compares five different ways of interacting with an attribute-based recommender system and shows that different types of users prefer different interaction methods. In an online experiment with an energy-saving recommender system the interaction methods are compared in terms of perceived control, understandability, trust in the system, user interface satisfaction, system effectiveness and choice satisfaction. The comparison takes into account several user characteristics, namely domain knowledge, trusting propensity and persistence. The results show that most users (and particularly domain experts) are most satisfied with a hybrid recommender that combines implicit and explicit preference elicitation, but that novices and maximizers seem to benefit more from a non-personalized recommender that just displays the most popular items.

As recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also "online" with test users to ensure good user experience. The user evaluation of recommenders is however complex and resource-consuming. We introduce a pragmatic procedure to evaluate recommender systems for experience products with test users, within industry constraints on time and budget. Researchers and practitioners can employ our approach to gain a comprehensive understanding of the user experience with their systems.

No abstract available.

Even though people are attracted by large, high quality recommendation sets, psychological research on choice overload shows that choosing an item from recommendation sets containing many attractive items can be a very difficult task. A web-based user experiment using a matrix factorization algorithm applied to the MovieLens dataset was used to investigate the effect of recommendation set size (5 or 20 items) and set quality (low or high) on perceived variety, recommendation set attractiveness, choice difficulty and satisfaction with the chosen item. The results show that larger sets containing only good items do not necessarily result in higher choice satisfaction compared to smaller sets, as the increased recommendation set attractiveness is counteracted by the increased difficulty of choosing from these sets. These findings were supported by behavioral measurements revealing intensified information search and increased acquisition times for these large attractive sets. Important implications of these findings for the design of recommender system user interfaces will be discussed.

In a recommender system that suggests options based on user attribute weights, the method of preference elicitation (PE) employed by a recommender system can influence users' satisfaction with the system, as well as the perceived usefulness and the understandability of the system. Specifically, we hypothesize that users with different levels of domain knowledge prefer different types of PE. While domain experts reported higher satisfaction and perceived usefulness with attribute-based PE (i.e., indicating preference levels for the domain-related attributes), novices preferred case-based PE (i.e., indicating the preference for specific examples, from which attribute-preferences can then be implicitly calculated). The paper discusses the decision-theoretical principles that are believed to lead to this distinction, as well as an experiment that provides substantial evidence for the hypothesis. Consequently, we introduce the idea of adapting the method of PE to users' domain knowledge on the fly using click stream data.