Whether choosing teammates for a project or partners for everyday life tasks, people constantly decide with whom to work. However, in these decisions, they often overemphasize characteristics that are not directly relevant to task performance. For example, prioritizing a partner’s trustworthiness for a task where competence is more important for good task performance. Artificial intelligence (AI) systems have the potential to mitigate these judgment errors by guiding decision-makers toward placing greater weight on traits that are more predictive of success for the specific task at hand. Although the potential usefulness of such systems is evident, previous work leaves unclear under what conditions and for what type of AI support people are willing to rely on and trust AI systems for such relational decisions (i.e., selecting a collaboration partner). To bridge this gap, our study examined how different forms of AI support shape users’ perceptions of the AI’s intellectual and social capabilities, their sense of autonomy, and their willingness to rely on and trust in AI when selecting a partner for a collaborative task. To do this, a total of 397 participants designed ideal partners for two collaborative tasks while receiving one of three forms of AI support: (1) recommendation, (2) explanation, or (3) knowledge nudges. This was tested in two different tasks: a competency-based task and a trustworthiness-based task. We found that richer AI support (through explanations or nudges) enhances perceived AI’s social and intellectual capabilities, but not autonomy. Perceptions of intellectual capabilities, rather than social capabilities, predict greater reliance. Both perceptions of AI capabilities mediate the effect of the type of AI support on reliance. Overall, the study advances understanding of human–AI collaboration by revealing how AI design features shape user perceptions and reliance when users need to evaluate and select their collaborators.

Machine learning-based automated essay scoring (AES) and feedback generation (AFG) tools have been developed since the 1960s, with some commercially deployed. Such systems are expected to lighten the labor-intensive task of essay scoring and help provide timely feedback to students. However, these tools have not been used effectively enough in practice despite the long history of this research field. We aim to determine how accurately currently available models can make the necessary corrections and detect improvable segments of text. Latest attempts of AFG utilize state-of-the-art generative large language models (LLMs) for writing evaluation. To the best of our knowledge, however, none of the past studies have included generative LLM-based models in a fine-grained sentence-to-sentence comparison with human feedback or among AI tools. To fill these gaps, we conduct an experimental comparison of human feedback and three AI tools developed at different stages of technological advancement. Findings indicate that the overlap between human and AI feedback is predominantly limited to surface-level linguistic features and that generative AI-augmented tools demonstrate a markedly higher capability than a tool based on conventional rule-based AI.

Research on how people experience music emphasizes the importance of exploration and diversity in listening. However, music recommender systems struggle with facilitating exploration. Even when music recommender systems are able to recommend something valuable to users that is outside their typical preferences, it still remains difficult to spark their interest. This paper presents a user study examining the efficacy of immersive and informative introductions in stimulating interest in songs that are beyond one’s usual preferences, an experience called Taste-Broadening Serendipity. We uncover two important mechanisms behind the effect of introductions: transportation and cognitive elaboration. Our findings indicate that transportation (i.e., being absorbed into a narrative world) is the strongest predictor of Taste-Broadening Serendipity, while cognitive elaboration (i.e., learning something new about the artist or social context in which the music emerged) has a weaker effect but is easier to stimulate. We propose that song introductions can play an important role in facilitating exploration and increasing diversity of listening on music streaming platforms.

The adoption of Recommender Systems (RSs) in various domains has become increasingly popular, but concerns have been raised about their lack of transparency and interpretability. While significant advancements have been made in creating explainable RSs, there is still a shortage of automated approaches that can deliver meaningful and contextual human-centered explanations. Numerous studies have evaluated explanations based on human-generated recommendations and explanations to address this gap. However, such approaches do not scale for real-world systems. Building on recent research that exploits Large Language Models (LLMs) for RSs, we propose leveraging the conversational capabilities of ChatGPT to provide users with personalized, human-like, and meaningful explanations for recommended items. Our article presents a user study with 94 participants that measures users’ perceptions of ChatGPT-generated explanations when acting as a recommender system. Regarding recommendations, we assess whether users prefer ChatGPT over random (but popular) recommendations. Concerning explanations, we assess users’ perceptions of personalization, effectiveness, and persuasiveness. We also break down the explanations in its constituent arguments and investigate the differences in argument types between generic and user-specific explanations. Our results show that participants rated ChatGPT’s recommendations significantly higher than random ones ( \(\beta=-.53\) , \({\textrm{p}} < .001\) ). Surprisingly, user-based explanations that explicitly referenced participants’ preferences were not perceived as more personalized or persuasive than generic ones, except when the recommended movie was unfamiliar, where user-based explanations became more effective ( \(\beta=0.35\) , \({\textrm{p}} < .05\) ). Overall, our findings highlight both the promise and the limitations of using LLMs for generating personalized explanations, suggesting that personalization is most impactful when users lack prior knowledge of the item.

While research on trust in human-AI interactions is gaining recognition, much work is conducted in lab settings that, therefore, lack ecological validity and often omit the trust development perspective. We investigated a real-world case in which logistics experts had worked with an AI system for several years (in some cases since its introduction). Through thematic analysis, three key themes emerged: First, although experts clearly point out AI system imperfections, they still showed to develop trust over time. Second, however, inconsistencies and frequent efforts to improve the AI system disrupted trust development, hindering control, transparency, and understanding of the system. Finally, despite the overall trustworthiness, experts overrode correct AI decisions to protect their colleagues’ well-being. By comparing our results with the latest trust research, we can confirm empirical work and contribute new perspectives, such as understanding the importance of human elements for trust development in human-AI scenarios.

Research in Explainable AI (XAI) has shown that explanations can improve users’ understanding of AI models, improve user performance and potentially reduce overreliance on AI predictions. However, this is mostly evaluated by static rather than dynamic measures, and the role of XAI on learning over trials is rarely studied. In this study, we use a context-free sequence prediction task, in which 458 participants predict the next symbol in a fixed sequence (with some noise) over 80 trials. We compare performance with AI and XAI advice against no AI support, and subsequently we test for learning by taking away the AI support after 40 trials (i.e., a reversal study design). Our results show that users learn faster with XAI than with AI without explanations or no AI and are better able to recover in performance from the removal of AI. However, the benefits of XAI on learning are much smaller for more difficult tasks. This work demonstrates the benefits of repeated measures user studies and multilevel modeling to better understand learning processes in XAI. It also shows the potential of AI explanations to help users to learn and poses XAI design suggestions to support learning in human-AI collaboration.

The 12th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems (IntRS'25) takes a user-centric perspective on recommender systems research. It brings together an interdisciplinary community of researchers and practitioners who explore a range of important issues on the "user side"of recommender systems, including user studies, psychology-informed design of RSs, novel interfaces, and evaluation methodologies. Its purpose is to identify critical challenges and emerging topics with a strong focus on fundamental historical challenges in the field. In this summary, we introduce the motivation and perspective of the workshop, review its history, and discuss the most critical issues that deserve attention for future research directions.

Complementing human decision-making with AI advice offers substantial advantages. However, humans do not always trust AI advice appropriately and are overly sensitive to incidental AI errors, even in cases with overall good performance. Today’s research still needs to uncover the underlying aspects of trust decline and recovery over time in repeated human-AI interactions. Our work investigates the consequences of incidental AI error on (self-reported) trust and participants’ reliance on AI advice. Results from our experiment, where 208 participants evaluated 14 legal cases before and after receiving algorithmic advice, showed that trust significantly decreased after early and late errors but was rapidly restored in both scenarios. Reliance significantly dropped only for early errors but not for late errors. In both scenarios, reliance was able to be restored. Results suggest that late (compared to early) errors are less drastic in trust loss and allow quicker recovery. These findings align with an interpretation in which humans can build up trust over time if a system is performing well, making them more tolerant of incidental AI errors.

Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners’ next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users’ domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.

The adoption of recommender systems (RSs) in various domains has become increasingly popular, but concerns have been raised about their lack of transparency and interpretability. While significant advancements have been made in creating explainable RSs, there is still a shortage of automated approaches that can deliver meaningful and contextual human-centered explanations. Numerous researchers have evaluated explanations based on human-generated recommendations and explanations to address this gap. However, such approaches do not scale for real-world systems. Building on recent research that exploits Large Language Models (LLMs) for RSs, we propose leveraging the conversational capabilities of ChatGPT to provide users with personalized, human-like, and meaningful explanations for recommended items. Our paper presents one of the first user studies that measure users’ perceptions of ChatGPT-generated explanations while acting as an RS. Regarding recommendations, we assess whether users prefer ChatGPT over random (but popular) recommendations. Concerning explanations, we assess users’ perceptions of personalization, effectiveness, and persuasiveness. Our findings reveal that users tend to prefer ChatGPT-generated recommendations over popular ones. Additionally, personalized rather than generic explanations prove to be more effective when the recommended item is unfamiliar.

The primary goal of Recommender Systems is to suggest the most suitable items to a user, aligning them with the user’s interests and needs. RSs are essential for modern e-commerce, helping users discover content and products by predicting suitable items based on their past behavior. However, their success isn’t just about advanced algorithms. The design of the user interface and a good integration with the human decision-making process are equally crucial. A well-designed interface enhances the user experience and makes recommendations more effective, while a poor interface can lead to frustration. Recognizing this limitation, recent trends in Recommender Systems (RSs) are increasingly focusing on integrating Symbiotic Human-Machine Decision-Making models. These models aim to offer users a dynamic and persuasive interface that helps them better understand and engage with recommendations. This shift is a crucial step toward developing recommender systems that truly connect with users and offer a more enjoyable, trustworthy, explainable, and user-friendly experience. Although early efforts concentrated on creating systems that could proactively predict user preferences and needs, modern RSs also emphasize the importance of providing users with control and transparency over their recommendations. Finding the right balance between proactivity and user control is essential to ensure that the system supports users without being too intrusive, thus improving their overall satisfaction. As Large Language Models (LLMs) become more integrated into recommender systems, the importance of user-centric interfaces and a deep understanding of decision-making becomes even more critical. Effective integration of LLMs requires interfaces that are both visually and cognitively engaging.

People are increasingly interacting with AI systems, but successful interactions depend on people trusting these systems only when appropriate. Since neither gaining trust in AI advice nor restoring lost trust after AI mistakes is warranted, we seek to better understand the development of trust and reliance in sequential human-AI interaction scenarios. In a 2 \({\times}\) 2 between-subject simulated AI experiment, we tested how model accuracy (high vs. low) and explanation type (human-like vs. abstract) affect trust and reliance on AI advice for repeated interactions. In the experiment, participants estimated jail times for 20 criminal law cases, first without and then with AI advice. Our results show that trust and reliance are significantly higher for high model accuracy. In addition, reliance does not decline over the trial sequence, and trust increases significantly with high accuracy. Human-like (vs. abstract) explanations only increased reliance on the high-accuracy condition. We furthermore tested the extent to which trust and reliance in a trial round can be explained by trust and reliance experiences from prior rounds. We find that trust assessments in prior trials correlate with trust in subsequent ones. We also find that the cumulative trust experience of a person in all earlier trial rounds correlates with trust in subsequent ones. Furthermore, we find that the two trust measures, trust and reliance, impact each other: prior trust beliefs not only influence subsequent trust beliefs but likewise influence subsequent reliance behavior, and vice versa. Executing a replication study yielded comparable results to our original study, thereby enhancing the validity of our findings.

We present a proof of concept application for interactive recommendations and explanations leveraging the capabilities of Large Language Models (LLMs). The application creates a highly interactive user-driven setting for recommendations giving users the possibility to explicitly tailor recommendations to their needs. Using the possibilities brought by LLMs, the application further generates convincing explanations of recommendations, aligned with the explicitly stated goals of the users. The web application continuously improves by incorporating user feedback and updating recommendations and explanations as needed.

Humans increasingly interact with AI systems, and successful interactions rely on individuals trusting such systems (when appropriate). Considering that trust is fragile and often cannot be restored quickly, we focus on how trust develops over time in a human-AI-interaction scenario. In a 2x2 between-subject experiment, we test how model accuracy (high vs. low) and type of explanation (human-like vs. not) affect trust in AI over time. We study a complex decision-making task in which individuals estimate jail time for 20 criminal law cases with AI advice. Results show that trust is significantly higher for high-accuracy models. Also, behavioral trust does not decline, and subjective trust even increases significantly with high accuracy. Human-like explanations did not generally affect trust but boosted trust in high-accuracy models.

Recommender systems (RSs) have undoubtedly played a significant role in addressing the information overload problem by efficiently filtering and suggesting relevant items to users. These systems use both explicit and implicit user preferences to filter available data and suggest items that might align with the user’s interests. This can range from recommending movies on a streaming platform based on previous views to suggesting products for purchase based on browsing history. In their early stages, RSs focused on enhancing their algorithmic capabilities to provide accurate recommendations. However, the overemphasis on algorithms resulted in neglecting the human aspect of the user experience. Recognizing this limitation, recent trends in RSs have started to shift their attention toward incorporating Symbiotic Human-Machines Decision Making models. These models aim to provide users with dynamic and persuasive interfaces that empower them to understand and engage better with the recommendations. This shift represents an essential step in creating recommender systems that truly resonate with users and create a more enjoyable, trustable, and user-friendly experience. A crucial aspect of recommender systems’ evolution lies in their proactive nature. Early works focused on designing systems that could proactively anticipate user preferences and needs. While this remains a valuable trait, modern RSs also recognize the importance of giving users control and transparency over their recommendations. Striking the right balance between proactivity and user control ensures that the system supports users without being overly intrusive, thus enhancing their overall satisfaction. These aspects are the main discussion topics of the Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at RecSys’23. In this summary, we introduce the workshop’s motivation and view, review its history, and discuss the most critical issues that deserve attention for future research directions.

Recommender systems are efficient at predicting users’ current preferences, but how users’ preferences develop over time is still under-explored. In this work, we study the development of users’ musical preferences. Exploring musical preference consistency between short-term and long-term preferences in data from earlier studies, we find that users with higher musical expertise have more consistent preferences for their top-listened artists and tags than those with lower musical expertise, while high expertise users also show more diverse listening behavior. Users typically chose to explore genres that were close to their current preferences, and this effect was stronger for expert users. Based on these findings, we conducted a user study on genre exploration to investigate (1) whether it is possible to nudge users to explore more distant genres, (2) how users’ exploration behavior within a genre is influenced by default recommendation settings that balance personalization with genre representativeness in different ways, and (3) how nudging for exploration increases the perceived helpfulness of recommendations for users to explore. Our results show that users were more likely to select the more distant genres if these genres were nudged by presented at the top of the list, however, users with high musical expertise were less likely to do so. We also find that the more representative default slider, which by default recommended more genre-representative tracks, made users set the slider at a less personalized level. The more representative slider position alone did not promote more exploration of users’ current preferences but when combined with the nudge for distant genres, the more representative slider positions effectively nudged users to explore. Nudging to explore does not necessarily lead to an increase in perceived helpfulness. On the one hand, nudging for exploration improves the perceived helpfulness for exploration by making users explore away from their current preferences. On the other hand, it reduces helpfulness due to lower perceived personalization as users move away from their personalized recommendations. To improve perceived helpfulness, it seems necessary to provide a balanced trade-off between exploration and personalization.

Model-agnostic explainable AI tools explain their predictions by means of ’local’ feature contributions. We empirically investigate two potential improvements over current approaches. The first one is to always present feature contributions in terms of the contribution to the outcome that is perceived as positive by the user (“positive framing”). The second one is to add “semantic labeling”, that explains the directionality of each feature contribution (“this feature leads to +5% eligibility”), reducing additional cognitive processing steps. In a user study, participants evaluated the understandability of explanations for different framing and labeling conditions for loan applications and music recommendations. We found that positive framing improves understandability even when the prediction is negative. Additionally, adding semantic labels eliminates any framing effects on understandability, with positive labels outperforming negative labels. We implemented our suggestions in a package ArgueView[11].

The constant increase in the amount of data and information available on the Web has made the development of systems that can support users in making relevant decisions increasingly important. Recommender systems (RSs) have emerged as tools to address this task. RSs use the preferences expressed by a user, either explicitly or implicitly, to filter the available information and proactively suggest items that might be of interest to him or her. Although in early works about the topic there was a strong interest in ways to make such systems proactive, user-friendly, and persuasive, over time they became increasingly focused on the algorithmic component solely. However, this trend is gradually being reversed and always more attention is nowadays placed also on Human Decision Making models that focus on supporting the end user in understanding what is being proposed through RSs by using dynamic and persuasive interfaces. A recommender system should be based on valuable strategies for proactively guiding users to items that match their preferences and therefore should put attention on how it is possible to make this process trustable, pleasant, and user-friendly. Such systems, moreover, should take into account psychological, cognitive and emotional aspects to enable personalization that is appropriate not only to the context of use but also to the psychological reactions of the end user. The workshop provides a venue for works that invest in the design of recommender systems which consider users’ experience during the interaction, as well as for works that explore the implications of human-computer interactions with different theories of human decision-making. In this summary, we introduce the Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at RecSys’22, review its history, and discuss the most important topics considered at the workshop.

Previous studies on exploration have shown that users can be nudged to explore further away from their current preferences. However, these effects were shown in a single session study, while it often takes time to explore new tastes and develop new preferences. In this work, we present a longitudinal study on users’ exploration behavior and behavior change over time after they have used a music genre exploration tool for four sessions in six weeks. We test two relevant nudges to help them explore more: the starting point (the personalization of the default initial playlist) and the visualization of users’ previous position(s). Our results show that the personalization level of the default initial playlist in the first session influences the preferred personalization level users set in the second session but fades away in later sessions as users start exploring in different directions. Visualization of users’ previous positions did not anchor users to stay closer to the initial defaults. Over time, users perceived the playlist to be more personalized to their tastes and helpful to explore the genre. Perceived helpfulness increased more when users explored further away from their current preferences. Apart from differences in self-reported measures, we also find some objective evidence for preference change in users’ top tracks from their Spotify profile, that over the period of 6 weeks moved somewhat closer to the genre that users selected to explore with the tool.

Research has demonstrated that users’ heuristic decision-making processes cause external factors like defaults and framing to influence the outcome of privacy decisions. Proponents of “privacy nudging” have proposed leveraging these effects to guide users’ decisions. Our research shows that defaults and framing not only influence the outcome of privacy decisions, but also the process of evaluating the contextual factors associated with the decision, effectively making the decision-making process more heuristic. In our analysis of an existing dataset of scenario-based smart home privacy decisions, we demonstrate that defaults and framing not only have a direct effect on participants’ decisions; they also moderate the effect of their cognitive appraisals of the presented scenarios on the decision. These results suggest that nudges like defaults and framing exacerbate the well-researched problem that people often employ heuristics rather than making deliberate privacy decisions, and that privacy-setting interfaces should avoid the effects of heuristic decision-making.

Recommender systems can be used to help users discover novel items and explore new tastes, for example in music genre exploration. However, little work has studied how to improve users’ understandability and acceptance of the novel items as well as support users to explore a new domain. In this paper, we investigate how two different visualizations and mood control influence the perceived control, informativeness and understandability of a music genre exploration tool, and further to improve the helpfulness for new music genre exploration. Specifically, we compare a bar chart visualization used by earlier work to a contour plot which allows users to compare their musical preferences with both the recommended tracks as well as the new genre. Mood control is implemented with two sliders to set a preferred mood on energy and valence features (that correlate with psychological mood dimensions). In the online user study, mood control was manipulated between subjects, and the visualizations were compared within subjects. During the study (N=102), we measured users’ subjective perceptions, experiences and the interactions with the system. Our results show that the contour plot visualization is perceived more helpful to explore new genres than the bar chart visualization, as the contour plot is perceived to be more informative and understandable. Users spent significantly more time and used the mood control more in the contour plot than in the bar chart visualization. Overall, our results show that the contour plot visualization combined with mood control serves as the most helpful way for new music genre exploration, because the mood control is easier to understand and use when made transparent via an informative visualization.

Recommender systems were originally developed as interactive intelligent systems that can proactively guide users to items that match their preferences. Despite its origin on the crossroads of HCI and AI, the majority of research on recommender systems gradually focused on objective accuracy criteria paying less and less attention to how users interact with the system as well as the efficacy of interface designs from users’ perspectives. This trend is reversing with the increased volume of research that looks beyond algorithms, into users’ interactions, decision making processes, and overall experience. The series of workshops on Interfaces and Human Decision Making for Recommender Systems focuses on the ”human side” of recommender systems. The goal of the research stream featured at the workshop is to improve users’ overall experience with recommender systems by integrating different theories of human decision making into the construction of recommender systems and exploring better interfaces for recommender systems. In this summary, we introduce the Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at RecSys’21, review its history, and discuss most important topics considered at the workshop.

Recommender systems are efficient at predicting users’ current preferences, but how users’ preferences develop over time is still under-explored. In this work, we study the development of users’ musical preferences. Exploring musical preference consistency between short-term and long-term preferences in data from earlier studies, we find that users with higher musical expertise have more consistent preferences at their top-listened artists and tags than those with lower musical expertise. Users typically chose to explore genres that were close to their current preferences, and this effect was stronger for expert users. Based on these findings we conducted a user study on genre exploration to investigate (1) whether it is possible to nudge users to explore more distant genres, and (2) how users’ exploration behaviors within a genre are influenced by default recommendation settings that balance personalization with genre representativeness in different ways. Our results show that users were more likely to select the more distant genres if these were presented at the top of the list. However, users with high musical expertise were less likely to do so, consistent with our earlier findings. When given a representative or mixed (balanced) default for exploration within a genre, users selected less personalized recommendation settings and explored further away from their current preferences, than with a personalized default. However, this effect was moderated by users’ slider usage behaviors. Overall, our results suggest that (personalized) defaults can nudge users to explore new, more distant genres and songs. However, the effect is smaller for those with higher musical expertise levels.

How can recommender interfaces help users to adopt new behaviors? In the behavioral change literature, social norms and other nudges are studied to understand how people can be convinced to take action (e.g., towel re-use is boosted when stating that “75% of hotel guests” do so), but most of these nudges are not personalized. In contrast, recommender systems know what to recommend in a personalized way, but not much human-computer interaction ( HCI ) research has considered how personalized advice should be presented to help users to change their current habits. We examine the value of depicting normative messages (e.g., “75% of users do X”), based on actual user data, in a personalized energy recommender interface called “Saving Aid.” In a study among 207 smart thermostat owners, we compared three different normative explanations (“Global.” “Similar,” and “Experienced” norm rates) to a non-social baseline (“kWh savings”). Although none of the norms increased the total number of chosen measures directly, we show that depicting high peer adoption rates alongside energy-saving measures increased the likelihood that they would be chosen from a list of recommendations. In addition, we show that depicting social norms positively affects a user’s evaluation of a recommender interface.

How can recommender interfaces help users to adopt new behaviors? In the behavioral change literature, nudges and norms are studied to understand how to convince people to take action (e.g. towel re-use is boosted when stating that `75% of hotel guests' do so), but what is advised is typically not personalized. Most recommender systems know what to recommend in a personalized way, but not much research has considered how to present such advice to help users to change their current habits. We examine the value of presenting normative messages (e.g. `75% of users do X') based on actual user data in a personalized energy recommender interface called `Saving Aid'. In a study among 207 smart thermostat owners, we compared three different normative explanations (`Global', `Similar', and `Experienced' norm rates) to a non-social baseline (`kWh savings'). Although none of the norms increased the total number of chosen measures directly, we show evidence that the effect of norms seems to be mediated by the perceived feasibility of the measures. Also, how norms were presented (i.e. specific source, adoption rate) affected which measures were chosen within our Saving Aid interface.

As an interactive intelligent system, recommender systems are developed to give recommendations that match users’ preferences. Since the emergence of recommender systems, a large majority of research focuses on objective accuracy criteria and less attention has been paid to how users interact with the system and the efficacy of interface designs from users’ perspectives. The field has reached a point where it is ready to look beyond algorithms, into users’ interactions, decision making processes, and overall experience. The series of workshops on Interfaces and Human Decision Making for Recommender Systems focuses on the ”human side” of recommender systems. The goal of the research stream featured at the workshop is to improve users’ overall experience with recommender systems by integrating different theories of human decision making into the construction of recommender systems and exploring better interfaces for recommender systems. In this summary, we introduce 7th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems at RecSys’20, review its history, and discuss most important topics considered at the workshop.

Previous studies have observed an intention-behavior gap that has been labeled the “privacy paradox”: people disclose personal information (behavior) despite expressing negative sharing intentions (in surveys). However, this phenomenon has not been studied in the Internet of Things (IoT) in which users’ personal information sharing is crucial for the functionality of the technology. We explore this phenomenon by comparing participants’ intentions (via a survey) with their actual behavior (via a privacy-setting interface) and controlling the data sharing device and storage. Furthermore, we explore the decision processes underlying these privacy decisions by measuring and manipulating these processes using an aspect listing task. We find a reversed intention-behavior gap in IoT: participants disclosed less (rather than more) information in the behavior condition than in the intention condition, an effect that was associated with fewer benefits than risk aspects listed in the behavior condition. The number and type of aspects listed fully mediated the effect of decision type (intention versus behavior) on the decision, which suggests that a risk-benefit calculation guided the privacy decision-making. Moreover, this reversed intention-behavior gap vanishes if we specifically ask participants to think about positive and negative aspects of the decision, as this allows them to consider both risks and benefits, irrespective of decision type.

Rapid innovations in electronic healthcare and behavior tracking systems are challenging health coaches (dietitians, personal trainers, etc.) to rethink their traditional roles and healthcare practices. At the same time, many current e-coaching systems have been developed without explicitly incorporating the healthcare professionals' perspective into the design process. In the current paper, we present three consecutive qualitative studies, starting from the health coach's perspective on successful coaching, progressively zooming in on the potential role and impact of technology as part of the coaching process. Our main finding is that coaches are concerned that introducing technology in the coaching process puts too much emphasis on behavioral information, lowering the attention for the client's lived experience, while understanding those experiences is key for successful coaching. We summarize our insights in a multi-channel communication model and draw implications for the design of supporting technology in health coaching.

Choosing adequate goals plays is central to the success of a task. With this study, we investigate tailoring the goals of a nutrition assistance system to the user's abilities according to a Rasch scale. To that end, we evaluated two versions of a mobile system that offers dietary tracking, visual feedback, and personalized recipe recommendations. The original version targets optimal nutritional behavior and focuses on the six least optimal nutrients (N=51). The adapted version targets only improved nutritional behavior compared to the status quo and thus tailors the advice to the next six achievable nutrients according to a Rasch scale (N=47). Results of the two-week study indicate that the tailored advice leads to higher success for the focused nutrients, and is perceived to be more diverse and personalized, and thus more effective.

As an interactive intelligent system, recommender systems are developed to give recommendations that match users' preferences. Since the emergence of recommender systems, a large majority of research focuses on objective accuracy criteria and less attention has been paid to how users interact with the system and the efficacy of interface designs from users' perspectives. The field has reached a point where it is ready to look beyond algorithms, into users' interactions, decision making processes, and overall experience. This workshop will focus on the "human side" of recommender systems research. The workshop goal is to improve users' overall experience with recommender systems by integrating different theories of human decision making into the construction of recommender systems and exploring better interfaces for recommender systems.

User-system interaction in recommender systems involves three aspects: temporal browsing (viewing recommendation lists and/or searching/filtering), action (performing actions on recommended items, e.g., clicking, consuming) and inaction (neglecting or skipping recommended items). Modern recommenders build machine learning models from recordings of such user interaction with the system, and in doing so they commonly make certain assumptions (e.g., pairwise preference orders, independent or competitive probabilistic choices, etc.). In this paper, we set out to study the effects of these assumptions along three dimensions in eight different single models and three associated hybrid models on a user browsing data set collected from a real-world recommender system application. We further design a novel model based on recurrent neural networks and multi-task learning, inspired by Decision Field Theory, a model of human decision making. We report on precision, recall, and MAP, finding that this new model outperforms the others.

Most recommender systems generate recommendations to match the user's current preference. However, users sometimes might have the goal to develop new preferences away from their current preference and use the recommender to guide them towards it. In this paper, we asked users to select a new genre to explore and studied what kind of recommendations would be more helpful for users to start exploring this new music taste. Three different recommendation methods are tested: one non-personalized which recommends the most representative tracks of the genre, one personalized method which considers songs from the new genre that best matches users' current preferences, and one mixed method which makes a trade-off between the two approaches. A comparative design was used in a user experiment in which participants were asked to evaluate the differences between the personalized method/mixed method and the non-personalized baseline. The mixed method results in recommendations that are more accurate and representative for the new genre than the personalized method. Users' perceived helpfulness for exploring the new genre is positively related to both perceived accuracy and perceived representativeness of the recommended items. Besides, recommendations from the mixed method are perceived more helpful for users high on Musical Sophistication Index for Active Engagement (MSAE). To our knowledge, this is one of the first studies using a recommender system to support users' preference development, and provides insights in how recommender systems can help users attain new goals and tastes.

As an interactive intelligent system, recommender systems are developed to give recommendations that match users' preferences. Since the emergence of recommender systems, a large majority of research focuses on objective accuracy criteria and less attention has been paid to how users interact with the system and the efficacy of interface designs from users' perspectives. The field has reached a point where it is ready to look beyond algorithms, into users' interactions, decision making processes, and overall experience. his workshop will focus on the "human side" of recommender systems research. The workshop goal is to improve users' overall experience with recommender systems by integrating different theories of human decision making into the construction of recommender systems and exploring better interfaces for recommender systems.

Temporally, users browse and interact with items in recommender systems. However, for most systems, the majority of the displayed items do not elicit any action from users. In other words, the user-system interaction process includes three aspects: browsing, action, and inaction. Prior recommender systems literature has focused more on actions than on browsing or inaction. In this work, we deployed a field survey in a live movie recommender system to interpret what inaction means from both the user's and the system's perspective, guided by psychological theories of human decision making. We further systematically study factors to infer the reasons of user inaction and demonstrate with offline data sets that this descriptive and predictive inaction model can provide benefits for recommender systems in terms of both action prediction and recommendation timing.

Current recommender systems often show the same most-highly recommended items again and again ignoring the feedback that users neither rate nor click on those items. We conduct an online field experiment to test two ways of manipulating top-N recommendations with the goal of improving user experience: cycling the top-N recommendation based on their past presentation and serpentining the top-N list mixing the best items into later recommendation requests. We find interesting tensions between opt-outs and activities, user perceived accuracy and freshness. Cycling within the same session might be a "love it or hate it" recommender property because users in it have a higher opt-out rate but engage in more activities. Cycling across sessions and serpentining increase user activities without significantly affecting opt-out rates. Users perceive more change and freshness but less accuracy and familiarity. Combining cycling and serpentining does not work as well as each individual manipulation separately. These two ways of manipulations on top-N list demonstrate some attractive properties but also call for innovative approaches to overcome their potential costs.

As intelligent interactive systems, recommender systems focus on determining predictions that fit the wishes and needs of users. Still, a large majority of recommender systems research focuses on accuracy criteria and much less attention is paid to how users interact with the system, and in which way the user interface has an influence on the selection behavior of the users. Consequently, it is important to look beyond algorithms. The main goals of the IntRS workshop are to analyze the impact of user interfaces and interaction design, and to explore human interaction with recommender systems from a human decision making perspective. Methodologies for evaluating these aspects are also within the scope of the workshop.

People often struggle to find appropriate energy-saving measures to take in the household. Although recommender studies show that tailoring a system's interaction method to the domain knowledge of the user can increase energy savings, they did not actually tailor the conservation advice itself. We present two large user studies in which we support users to make an energy-efficient behavioral change by presenting tailored energy-saving advice. Both systems use a one-dimensional, ordinal Rasch scale, which orders 79 energy-saving measures on their behavioral difficulty and link this to a user's energy-saving ability for tailored advice. We established that recommending Rasch-based advice can reduce a user's effort, increase system support and, in turn, increase choice satisfaction and lead to the adoption of more energy-saving measures. Moreover, follow-up surveys administered four weeks later point out that tailoring advice on its feasibility can support behavioral change.

As intelligent interactive systems, recommender systems focus on determining predictions that fit the wishes and needs of users. Still, a large majority of recommender systems research focuses on accuracy criteria and much less attention is paid to how users interact with the system, and in which way the user interface has an influence on the selection behavior of the users. Consequently, it is important to look beyond algorithms. The main goals of the IntRS workshop are to analyze the impact of user interfaces and interaction design, and to explore human interaction with recommender systems. Methodologies for evaluating these aspects are also within the scope of the workshop.

Behaviorism is the currently-dominant paradigm for building and evaluating recommender systems. Both the operation and the evaluation of recommender system applications are most often driven by analyzing the behavior of users. In this paper, we argue that listening to what users say about the items and recommendations they like, the control they wish to exert on the output, and the ways in which they perceive the system and not just observing what they do will enable important developments in the future of recommender systems. We provide both philosophical and pragmatic motivations for this idea, describe the various points in the recommendation and evaluation processes where explicit user input may be considered, and discuss benefits that may result from considered incorporation of user preferences at each of these points. In particular, we envision recommender applications that aim to support users' better selves: helping them live the life that they desire to lead. For example, recommender-assisted behavior change requires algorithms to predict not what users choose or do now, inferable from behavioral data, but what they should choose or do in the future to become healthier, fitter, more sustainable, or culturally aware. We hope that our work will spur useful discussion and many new ideas for recommenders that empower their users.

In this work we investigate the use of behavior feasibility to adapt and personalize lifestyle-targeting recommender systems for the prevention and treatment of hypertension. Based on survey data (N=300) we model the feasibiliy of 63 behaviors through a Rasch model, describing the engagement in a behavior as a function of the behavior's difficulty and the person's ability. We formulate two feasibility-tailored recommendation strategies that utilize the Rasch model. The engagement maximization strategy aims at maximizing the probability of engagement by proposing very feasible behaviors while the motivation maximization strategy aims to challenge users by matching the difficulty of the advice with the ability of the user, thereby maximizing motivation. In an online study (N=150) we assessed user preference for either strategies (embodied as virtual coaches) in comparison with a random control strategy. Our results show that coaches selecting feasible health advice resonate better with the patient than control. In general patients significantly preferred the engagement maximization strategy over random advice on most factors, while patients with a medium level of ability significantly preferred the motivation maximization strategy on all factors.

We investigate the usability of humanlike agent-based interfaces for interactive advice-giving systems. In an experiment with a travel advisory system, we manipulate the “humanlikeness” of the agent interface. We demonstrate that users of the more humanlike agents try to exploit capabilities that were not signaled by the system. This severely reduces the usability of systems that look human but lack humanlikehumanlike capabilities (overestimation effect). We explain this effect by showing that users of humanlike agents form anthropomorphic beliefs (a user's “mental model”) about the system: They act humanlike towards the system and try to exploit typical humanlike capabilities they believe the system possesses. Furthermore, we demonstrate that the mental model users form of an agent-based system is inherently integrated (as opposed to the compositional mental model they form of conventional interfaces): Cues provided by the system do not instill user responses in a one-to-one matter but are instead integrated into a single mental model.

People like variety and often prefer to choose from large item sets. However, large sets can cause a phenomenon called “choice overload”: they are more difficult to choose from, and as a result decision makers are less satisfied with their choices. It has been argued that choice overload occurs because large sets contain more similar items. To overcome this effect, the present paper proposes that increasing the diversity of item sets might make them more attractive and satisfactory, without making them much more difficult to choose from. To this purpose, by using structural equation model methodology, we study diversification based on the latent features of a matrix factorization recommender model. Study 1 diversifies a set of recommended items while controlling for the overall quality of the set, and tests it in two online user experiments with a movie recommender system. Study 1a tests the effectiveness of the latent feature diversification, and shows that diversification increases the perceived diversity and attractiveness of the item set, while at the same time reducing the perceived difficulty of choosing from the set. Study 1b subsequently shows that diversification can increase users’ satisfaction with the chosen option, especially when they are choosing from small, diverse item sets. Study 2 extends these results by testing our diversification algorithm against traditional Top-N recommendations, and finds that diverse, small item sets are just as satisfying and less effortful to choose from than Top-N recommendations. Our results suggest that, at least for the movie domain, diverse small sets may be the best thing one could offer a user of a recommender system.

We studied an alternative choice-based interface for preference elicitation during the cold start phase and compared it directly with a standard rating-based interface. In this alternative interface users started from a diverse set covering all movies and iteratively narrowed down through a matrix factorization latent feature space to smaller sets of items based on their choices. The results show that compared to a rating-based interface, the choice-based interface requires less effort and results in more satisfying recommendations, showing that it might be a promising candidate for alleviating the cold start problem of new users.

Recent developments in user evaluation of recommender systems have brought forth powerful new tools for understanding what makes recommendations effective and useful. We apply these methods to understand how users evaluate recommendation lists for the purpose of selecting an algorithm for finding movies. This paper reports on an experiment in which we asked users to compare lists produced by three common collaborative filtering algorithms on the dimensions of novelty, diversity, accuracy, satisfaction, and degree of personalization, and to select a recommender that they would like to use in the future. We find that satisfaction is negatively dependent on novelty and positively dependent on diversity in this setting, and that satisfaction predicts the user's final selection. We also compare users' subjective perceptions of recommendation properties with objective measures of those same characteristics. To our knowledge, this is the first study that applies modern survey design and analysis techniques to a within-subjects, direct comparison study of recommender algorithms.

A primary function of recommender systems is to help their users to make better choices and decisions. The overall goal of the workshop is to analyse and discuss novel techniques and approaches for supporting effective and efficient human decision making in different types of recommendation scenarios. The submitted papers discuss a wide range of topics from core algorithmic issues to the management of the human computer interaction.

One of the challenges for recommender systems is that users struggle to accurately map their internal preferences to external measures of quality such as ratings. We study two methods for supporting the mapping process: (i) reminding the user of characteristics of items by providing personalized tags and (ii) relating rating decisions to prior rating decisions using exemplars. In our study, we introduce interfaces that provide these methods of support. We also present a set of methodologies to evaluate the efficacy of the new interfaces via a user experiment. Our results suggest that presenting exemplars during the rating process helps users rate more consistently, and increases the quality of the data.

Many recommendation systems rely on explicit ratings provided by their users. Often these ratings are provided long after consuming the item, relying heavily on people's representation of the quality of the item in memory. This paper investigates a psychological process, the "positivity effect", that influences the retrieval of quality judgments from our memory by which pleasant items are being processed and recalled from memory more effectively than unpleasant items. In an offline study on the MovieLens data we used the time between release date and rating date as a proxy for the time between consumption and rating. Ratings for movies tend to increase over time, consistent with the positivity effect. A subsequent online user study used a direct measure of time between rating and consumption, by asking users to rate movies (recently aired on television) and to explicitly report how long ago they watched these movies. In contrast to the offline study we find that ratings tend to decline over time showing reduced accuracy in ratings for items experienced long ago. We discuss the impact these rating dynamics might have on recommender algorithms, especially in cases where a new user has to submit his preferences to a system.

Interacting with a recommender system means to take different decisions such as selecting an item from a recommendation list, selecting a specific item feature value (e.g., camera's size, zoom) as a search criteria, selecting feedback features to be critiqued in a critiquing based recommendation session, or selecting a repair proposal for inconsistent user preferences when interacting with a knowledge-based recommender. In all these situations, users face a decision task. This workshop ([email protected]) focuses on approaches for supporting effective and efficient human decision making in different types of recommendation scenarios.

Research on recommender systems typically focuses on the accuracy of prediction algorithms. Because accuracy only partially constitutes the user experience of a recommender system, this paper proposes a framework that takes a user-centric approach to recommender system evaluation. The framework links objective system aspects to objective user behavior through a series of perceptual and evaluative constructs (called subjective system aspects and experience, respectively). Furthermore, it incorporates the influence of personal and situational characteristics on the user experience. This paper reviews how current literature maps to the framework and identifies several gaps in existing work. Consequently, the framework is validated with four field trials and two controlled experiments and analyzed using Structural Equation Modeling. The results of these studies show that subjective system aspects and experience variables are invaluable in explaining why and how the user experience of recommender systems comes about . In all studies we observe that perceptions of recommendation quality and/or variety are important mediators in predicting the effects of objective system aspects on the three components of user experience: process (e.g. perceived effort, difficulty), system (e.g. perceived system effectiveness) and outcome (e.g. choice satisfaction). Furthermore, we find that these subjective aspects have strong and sometimes interesting behavioral correlates (e.g. reduced browsing indicates higher system effectiveness). They also show several tradeoffs between system aspects and personal and situational characteristics (e.g. the amount of preference feedback users provide is a tradeoff between perceived system usefulness and privacy concerns). These results, as well as the validated framework itself, provide a platform for future research on the user-centric evaluation of recommender systems.

This paper compares five different ways of interacting with an attribute-based recommender system and shows that different types of users prefer different interaction methods. In an online experiment with an energy-saving recommender system the interaction methods are compared in terms of perceived control, understandability, trust in the system, user interface satisfaction, system effectiveness and choice satisfaction. The comparison takes into account several user characteristics, namely domain knowledge, trusting propensity and persistence. The results show that most users (and particularly domain experts) are most satisfied with a hybrid recommender that combines implicit and explicit preference elicitation, but that novices and maximizers seem to benefit more from a non-personalized recommender that just displays the most popular items.

As recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also "online" with test users to ensure good user experience. The user evaluation of recommenders is however complex and resource-consuming. We introduce a pragmatic procedure to evaluate recommender systems for experience products with test users, within industry constraints on time and budget. Researchers and practitioners can employ our approach to gain a comprehensive understanding of the user experience with their systems.

No abstract available.

Even though people are attracted by large, high quality recommendation sets, psychological research on choice overload shows that choosing an item from recommendation sets containing many attractive items can be a very difficult task. A web-based user experiment using a matrix factorization algorithm applied to the MovieLens dataset was used to investigate the effect of recommendation set size (5 or 20 items) and set quality (low or high) on perceived variety, recommendation set attractiveness, choice difficulty and satisfaction with the chosen item. The results show that larger sets containing only good items do not necessarily result in higher choice satisfaction compared to smaller sets, as the increased recommendation set attractiveness is counteracted by the increased difficulty of choosing from these sets. These findings were supported by behavioral measurements revealing intensified information search and increased acquisition times for these large attractive sets. Important implications of these findings for the design of recommender system user interfaces will be discussed.

In a recommender system that suggests options based on user attribute weights, the method of preference elicitation (PE) employed by a recommender system can influence users' satisfaction with the system, as well as the perceived usefulness and the understandability of the system. Specifically, we hypothesize that users with different levels of domain knowledge prefer different types of PE. While domain experts reported higher satisfaction and perceived usefulness with attribute-based PE (i.e., indicating preference levels for the domain-related attributes), novices preferred case-based PE (i.e., indicating the preference for specific examples, from which attribute-preferences can then be implicitly calculated). The paper discusses the decision-theoretical principles that are believed to lead to this distinction, as well as an experiment that provides substantial evidence for the hypothesis. Consequently, we introduce the idea of adapting the method of PE to users' domain knowledge on the fly using click stream data.