Advances in Computer Vision have enabled large-scale analysis and comparison of visual art collections. Similarity, a central measure among image analysis, is commonly used to support corpus exploration or as a way to recommend relevant items. Yet, in cultural heritage contexts, it remains inherently multifaceted and context-dependent, creating ambiguity in how it is defined and operationalized across projects. This paper reframes similarity as an explicitly constructed, multifaceted, and situational measure. It introduces a nested model of similarity construction across five interdependent layers—computational accuracy, facet selection, facet integration, definition control, and perceptual alignment—showing how similarity emerges through the interaction of technical measurement, system design, and user judgment. The model provides a vocabulary for identifying how similarity is shaped, where mismatches arise. We instantiate the model in a prototype system and examine its implications on design and evaluation through a user study with 39 participants. The findings show that transparency and control support users in shaping and interpreting similarity, while differences between expert and non-expert users highlight the role of disciplinary context. Overall, the paper argues for a shift away from metric-focused similarity definitions, and toward a layered approach that recognizes interpretation and user agency.

Adaptive learning platforms can personalise learning in classrooms, yet teachers need tools to monitor student progress and rapidly identify students who need additional attention. Monitoring dashboards that detect outlier students can fulfil this need, but insufficient explanation of the detection logic may undermine trust and lead to inappropriate use. To address this challenge, we iteratively designed and implemented a real-time monitoring dashboard with model-centric and data-centric explanations, informed by teacher input. We then conducted a counterbalanced within-group experiment with follow-up interviews to study how 11 teachers used the dashboard in a real classroom and how they engaged with the explanations to calibrate their trust. We found that teachers successfully integrated the dashboard into their classes, and that their trust was shaped by many dispositional, situational, and learnt factors. Crucially, the data-centric explanations enabled teachers to validate the accuracy of outlier predictions, check alignment with their prior knowledge of students, and identify suitable interventions. Based on these findings, we present design recommendations for explainable outlier detection systems in education.

22q11.2 Deletion Syndrome (22q11 DS) is the most prevalent chromosomal microdeletion (∼ 1:4 500 live births), yet fewer than a third of parents feel equipped to discuss the condition with their child. Clinical encounters are brief and infrequent, suggesting that families need support that extends into daily life. We present GeneticPuzzle, a mobile psychoeducational application that digitises and extends a clinically validated physical puzzle tool used by over 50 families. Through two iterative studies, an expert evaluation and proof-of-concept with 8 families followed by a 6-month deployment with 43 families, we identified three design pillars: (1) interactive multimedia psychoeducation, (2) conversation scaffolding through a question logbook and puzzle-based recognition, and (3) longitudinal monitoring of evolving self-understanding. Our studies showcased the app’s ability to support family communication, with a supporting role in conversation scaffolding, and helping disease-related self-identification.

While substantial work has focused on explaining job–candidate matches in recruiting, a gap remains between these explanations and end-user goals, with visual explanations often being complex and overwhelming. Moreover, existing approaches are often static and provide limited interaction support, making it difficult for users to explore explanations in a way that matches their information needs. Guided by a human–centered design process, we present CAPTURE, a visual–conversational dashboard designed to support on-demand explanations in job–candidate matching. It integrates visual explanations with a conversational component powered by a multi-agent architecture that interprets user queries and orchestrates data retrieval and analysis workflows. By linking rich textual responses with relevant visualizations, CAPTURE enables flexible exploration and supports user-driven explanations in job–candidate matching algorithms.

Recommender systems are increasingly used in mobile health interventions, such as managing Chronic Musculoskeletal Pain (CMP). While researchers have highlighted the importance of explaining health-related recommendations to lay users, with benefits such as increased trust and a higher tendency to follow up on these recommendations, how to design explanations for lay users in critical contexts such as health remains largely unexplored. To address this gap, we develop a mobile health application to support users with CMP through coaching and personalised health recommendations delivered via a conversational rule-based recommender system. This article describes the three-phase iterative development of the RS, involving health experts and end users. In the first iteration, we conduct a preliminary validation study with \(N=282\) participants to ensure the app’s validity and improve the initial set of health recommendations. Next, two user studies are conducted centred around designing effective and understandable explanations for these recommendations. First, we design six explanation modalities tailored towards lay users, and through a qualitative study ( \(N=11\) ), extract initial design guidelines for explaining health recommendations, finding a strong preference towards feature importance explanations and identifying issues with modalities that highlight negative emotions. Given these results, we explore whether extending feature importance explanations with textual information into a ‘hybrid’ explanation could benefit end users, and whether these benefits depend on a user’s personal characteristics (need for cognition and ease-of-satisfaction). Through a mixed-methods study with \(N=262\) participants, we find that the hybrid modality significantly increased user trust, transparency, persuasiveness, usefulness and satisfaction compared to unimodal explanations. However, users with a higher need for cognition rate unimodal explanations more positively than hybrid ones.

Representation bias is one of the most common types of biases in artificial intelligence (AI) systems, causing AI models to perform poorly on underrepresented data segments. Although AI practitioners use various methods to reduce representation bias, their effectiveness is often constrained by insufficient domain knowledge in the debiasing process. To address this gap, this paper introduces a set of generic design guidelines for effectively involving domain experts in representation debiasing. We instantiated our proposed guidelines in a healthcare-focused application and evaluated them through a comprehensive mixed-methods user study with 35 healthcare experts. Our findings show that involving domain experts can reduce representation bias without compromising model accuracy. Based on our findings, we also offer recommendations for developers to build robust debiasing systems guided by our generic design guidelines, ensuring more effective inclusion of domain experts in the debiasing process.

This paper explores the adoption of a clinical decision support system (cDSS) utilizing game-based digital biomarkers for diagnosing mild cognitive impairment (MCI). Specifically, it investigates how different explanation methods, with a focus on data-centric explanations, impact perceived ease-of-use, perceived usefulness, and trust among healthcare professionals (HCPs). Through a qualitative study with 12 HCPs, we assess their interactions with an explainable AI (XAI)-enriched cDSS. The findings indicate that HCPs are open to adopting XAI-enriched cDSS to communicate the outcomes of game-based digital biomarkers. HCPs preferred to receive key diagnostic information in an easily digestible format. Both local explanations of intra-personal evolutionary data and global overview of normative data were found to be valuable for interpreting digital biomarkers. HCPs tended to trust the machine learning algorithms as a black box, but they considered the dataset used for training the model and the outcome prediction to be crucial. Therefore, presenting the uncertainty alongside the prediction was deemed important. These insights underscore the importance of designing cDSS tools that foster trust through clear, actionable explanations, paving the way for improved decision-making in clinical contexts.

E-learning platforms that personalise content selection with AI are often criticised for lacking transparency and controllability. Researchers have therefore proposed solutions such as open learner models and letting learners select from ranked recommendations, which engage learners before or after the AI-supported selection process. However, little research has explored how learners - especially adolescents - could engage during such AI-supported decision-making. To address this open challenge, we iteratively designed and implemented a control mechanism that enables learners to steer the difficulty of AI-compiled exercise series before practice, while interactively analysing their control's impact in a 'what-if' visualisation. We evaluated our prototypes through four qualitative studies involving adolescents, teachers, EdTech professionals, and pedagogical experts, focusing on different types of visual explanations for recommendations. Our findings suggest that 'why' explanations do not always meet the explainability needs of young learners but can benefit teachers. Additionally, 'what-if' explanations were well-received for their potential to boost motivation. Overall, our work illustrates how combining learner control and visual explanations can be operationalised on e-learning platforms for adolescents. Future research can build upon our designs for 'why' and 'what-if' explanations and verify our preliminary findings.

Counterfactual explanations offer actionable insights by illustrating how changes to inputs can lead to different outcomes.However, these explanations often suffer from ambiguity and impracticality, limiting their utility for non-expert users with limited AI knowledge.Augmenting counterfactual explanations with Large Language Models (LLMs) has been proposed as a solution, but little research has examined their benefits and challenges for non-experts.To address this gap, we developed a healthcare-focused system that leverages conversational AI agents to enhance counterfactual explanations, offering clear, actionable recommendations to help patients at high risk of cardiovascular disease (CVD) reduce their risk.Evaluated through a mixed-methods study with 34 participants, our findings highlight the effectiveness of agent-augmented counterfactuals in improving actionable recommendations.Results further indicate that users with prior experience using conversational AI demonstrated greater effectiveness in utilising these explanations compared to novices.Furthermore, this paper introduces a set of generic guidelines for creating augmented counterfactual explanations, incorporating safeguards to mitigate common LLM pitfalls, such as hallucinations, and ensuring the explanations are both actionable and contextually relevant for non-expert users.

Identifying explanation needs based on user characteristics has been the focus of human-centred research within XAI for some time.In Ribera et al. 's proposal of user-centred XAI, expertise was used as a proxy for characterising the user, and in turn guide explanation design.Since then, the research landscape has evolved to include a broader notion of stakeholders, ranging from AI developers to external regulators to the affected users of AI decisions.However, with this broadening of stakeholder roles, there emerged a pattern of conflating expertise and role, such as the term "end user" being used interchangeably for domain experts using (X)AI for decisionmaking and lay users impacted by AI decisions, with both having vastly different explanatory needs.In this work, we revisit previous surveys with the aim to identify and classify stakeholders in the XAI ecosystem.We propose to consistently categorise stakeholders along separate expertise and role dimensions.By disentangling both, we present a framework that highlights the diversity of stakeholder goals and the challenges of aligning explanation design with varied user requirements.Our analysis maps stakeholders onto these dimensions and discusses how using both expertise and role can inform the development of more tailored and effective XAI solutions.

sponsorship: We would like to thank ZAVO, and Joke Vandepitte in particular, for allowing us to collaborate. We would like to thank FWO by facilitating this interdisciplinary research. Additionally, we would like to thank all participants for their time and valuable insights. This research is part of the research projects funded by KU Leuven (grant C14/21/072) and the Flanders AI Research Program (FAIR). (KU Leuven|C14/21/072, Flanders AI Research Program (FAIR))

Explanations in interactive machine-learning systems facilitate debugging and improving prediction models. However, the effectiveness of various global model-centric and data-centric explanations in aiding domain experts to detect and resolve potential data issues for model improvement remains unexplored. This research investigates the influence of data-centric and model-centric global explanations in systems that support healthcare experts in optimising models through automated and manual data configurations. We conducted quantitative (n=70) and qualitative (n=30) studies with healthcare experts to explore the impact of different explanations on trust, understandability and model improvement. Our results reveal the insufficiency of global model-centric explanations for guiding users during data configuration. Although data-centric explanations enhanced understanding of post-configuration system changes, a hybrid fusion of both explanation types demonstrated the highest effectiveness. Based on our study results, we also present design implications for effective explanation-driven interactive machine-learning systems.

While 33.6% of college students suffer from mental health problems, only 24.6% of these students with symptoms would seek professional help due to their personal attitudes or costs associated with therapy. Psychotherapy chatbots may offer a solution as they are always available, anonymous, and cost-effective. Research has shown that these chatbots can significantly reduce symptoms of anxiety and depression. However, there is a lack of understanding about the personalization preferences of users and the effects of personalization on health outcomes. To investigate this, we developed a personalizable psychotherapy chatbot designed to provide personalized help. In a randomized controlled trial (n = 54), participants were either assigned to a personalizable condition or a non-personalizable control condition. After 1 week of usage, participants had a significantly higher therapeutic bond with the personalized version compared to the baseline. In fact, the therapeutic bond was similar to that between a psychologist and his client. This is a promising result, as a high therapeutic bond has been linked to therapeutic success in psychotherapy. Participants reported that the therapy style, personality, and avatar were the most important personalizable aspects of the chatbot. Participants also liked the chatbot’s usage of their name and the transparency about what the chatbot had learned about them. These features are likely important for establishing a strong therapeutic bond with users. However, the ability to personalize the chatbot had no impact on the usage intentions of the participants. This can be explained by the fact that users from both conditions equally reported that the chatbot was able to help them with their mental health. 53 participants also indicated that they would be willing to use a psychotherapy chatbot when integrated with a human therapist. These findings indicate the potential of psychotherapy chatbots and the need for further research on their integration with traditional psychotherapy.

Recent advancements in Educational AI have focused on models for automatic question generation. Yet, these advancements face challenges: (1) their "black-box" nature limits transparency, thereby obscuring the decision-making process; and (2) their novelty sometimes causes inaccuracies due to limited feedback systems. Explainable AI (XAI) aims to address the first limitation by clarifying model decisions, while Interactive Machine Learning (IML) emphasises user feedback and model refinement. However, both XAI and IML solutions primarily serve AI experts, often neglecting novices like teachers. Such oversights lead to issues like misaligned expectations and reduced trust. Following the user-centred design method, we collaborated with teachers and ed-tech experts to develop an AI-aided system for generating multiple-choice question distractors, which incorporates feedback, control, and visual explanations. Evaluating these through semi-structured interviews with 12 teachers, we found a strong preference for the feedback feature, enabling teacher-guided AI improvements. Control and explanations’ usefulness was largely dependent on model performance: they were valued when the model performed well. If the model did not perform well, teachers sought context over AI-centric explanations, suggesting a tilt towards data-centric explanations. Based on these results, we propose guidelines for creating tools that enable teachers to steer and interact with question-generating AI models.

Explainable artificial intelligence is increasingly used in machine learning (ML) based decision-making systems in healthcare. However, little research has compared the utility of different explanation methods in guiding healthcare experts for patient care. Moreover, it is unclear how useful, understandable, actionable and trustworthy these methods are for healthcare experts, as they often require technical ML knowledge. This paper presents an explanation dashboard that predicts the risk of diabetes onset and explains those predictions with data-centric, feature-importance, and example-based explanations. We designed an interactive dashboard to assist healthcare experts, such as nurses and physicians, in monitoring the risk of diabetes onset and recommending measures to minimize risk. We conducted a qualitative study with 11 healthcare experts and a mixed-methods study with 45 healthcare experts and 51 diabetic patients to compare the different explanation methods in our dashboard in terms of understandability, usefulness, actionability, and trust. Results indicate that our participants preferred our representation of data-centric explanations that provide local explanations with a global overview over other methods. Therefore, this paper highlights the importance of visually directive data-centric explanation method for assisting healthcare experts to gain actionable insights from patient health records. Furthermore, we share our design implications for tailoring the visual representation of different explanation methods for healthcare experts.

Researchers have widely acknowledged the potential of control mechanisms with which end-users of recommender systems can better tailor recommendations. However, few e-learning environments so far incorporate such mechanisms, for example for steering recommended exercises. In addition, studies with adolescents in this context are rare. To address these limitations, we designed a control mechanism and a visualisation of the control’s impact through an iterative design process with adolescents and teachers. Then, we investigated how these functionalities affect adolescents’ trust in an e-learning platform that recommends maths exercises. A randomised controlled experiment with 76 middle school and high school adolescents showed that visualising the impact of exercised control significantly increases trust. Furthermore, having control over their mastery level seemed to inspire adolescents to reasonably challenge themselves and reflect upon the underlying recommendation algorithm. Finally, a significant increase in perceived transparency suggested that visualising steering actions can indirectly explain why recommendations are suitable, which opens interesting research tracks for the broader field of explainable AI.

Due to their historical nature, humanistic data encompass multiple sources of uncertainty. While humanists are accustomed to handling such uncertainty with their established methods, they are cautious of visualizations that appear overly objective and fail to communicate this uncertainty. To design more trustworthy visualizations for humanistic research, therefore, a deeper understanding of its relation to uncertainty is needed. We systematically reviewed 126 publications from digital humanities literature that use visualization as part of their research process, and examined how uncertainty was handled and represented in their visualizations. Crossing these dimensions with the visualization type and use, we identified that uncertainty originated from multiple steps in the research process from the source artifacts to their datafication. We also noted how besides known uncertainty coping strategies, such as excluding data and evaluating its effects, humanists also embraced uncertainty as a separate dimension important to retain. By mapping how the visualizations encoded uncertainty, we identified four approaches that varied in terms of explicitness and customization. This work contributes with two empirical taxonomies of uncertainty and it's corresponding coping strategies, as well as with the foundation of a research agenda for uncertainty visualization in the digital humanities. Our findings further the synergy among humanists and visualization researchers, and ultimately contribute to the development of more trustworthy, uncertainty-aware visualizations.

Recommender systems are increasingly used in high-risk application domains, including healthcare. It has been shown that explanations are crucial in this context to support decision-making. This paper explores how to explain call recommendations to nursing home staff, providing insights into call priority, notifications, and resident information. We present the design and implementation of a recommender engine and a mobile application designed to support call recommendations and explain these recommendations that may contribute to residents’ safety and quality of care. More specifically, we report on the results of a user-centered design approach with residents (N=12) and healthcare professionals (N=4), and a final evaluation (N=12) after four months of deployment. The results show that our design approach provides a valuable tool for more accurate and efficient decision-making. The overall system encourages nursing home staff to provide feedback and annotate, resulting in more confidence in the system. We discuss usability issues, challenges, and reflections to be considered in future health recommender systems.

In the scope of explainable artificial intelligence, explanation techniques are heavily studied to increase trust in recommender systems. However, studies on explaining recommendations typically target adults in e-commerce or media contexts; e-learning has received less research attention. To address these limits, we investigated how explanations affect adolescents’ initial trust in an e-learning platform that recommends mathematics exercises with collaborative filtering. In a randomized controlled experiment with 37 adolescents, we compared real explanations with placebo and no explanations. Our results show that real explanations significantly increased initial trust when trust was measured as a multidimensional construct of competence, benevolence, integrity, intention to return, and perceived transparency. Yet, this result did not hold when trust was measured one-dimensionally. Furthermore, not all adolescents attached equal importance to explanations and trust scores were high overall. These findings underline the need to tailor explanations and suggest that dynamically learned factors may be more important than explanations for building initial trust. To conclude, we thus reflect upon the need for explanations and recommendations in e-learning in low-stakes and high-stakes situations.

Artificial intelligence (AI) has seen a steady increase in use in the health and medical field, where it is used by lay users and health experts alike. However, these AI systems often lack transparency regarding the inputs and decision making process (often called black boxes), which in turn can be detrimental to the user’s satisfaction and trust towards these systems. Explainable AI (XAI) aims to overcome this problem by opening up certain aspects of the black box, and has proven to be a successful means of increasing trust, transparency and even system effectiveness. However, for certain groups (i.e. lay users in health), explanation methods and evaluation metrics still remain underexplored. In this paper, we will outline our research regarding designing and evaluating explanations for health recommendations for lay users and domain experts, as well as list a few takeaways we were already able to find in our initial studies.

Artificial Intelligence (AI) supports many of our everyday activities and decisions. However, personalized algorithmic recommendations often produce adverse experiences due to a lack of awareness, control, or transparency. While research has directed solutions on graphical user interfaces (GUIs), there are no explorations of Tangible User Interfaces (TUIs) to improve the experience with such systems, despite the valid existing academic arguments in favor of this exploration. Therefore, centering on transparency and control, we analyzed how 18 users of movie recommender systems perceived four different TUIs using individual co-design sessions and post-interview questionnaires. Through thematic analysis, we identified seven design considerations while designing TUIs to interact with algorithmic movie recommender systems: (1) Distinctions between TUIs and GUIs; (2) TUIs replacing predominant interfaces; (3) Preference for single-device TUIs; (4) The relevance of granular control for TUIs; (5) Apparent transparency limitations of TUIs; (6) TUIs and algorithmic social computing; and (7) Overview of specific design choices, including advantages and disadvantages of soft, hard, rounded, cubic, and humanoid interfaces. These findings inspired Recffy: the first functional TUI designed to enhance awareness and control in personalized movie recommendations. Based on this study, we propose the concept of Tangible Algorithms: TUIs dedicated to enhancing the interaction of algorithmic systems and their profiling processes or decisions in a specific context. Furthermore, we describe the relevance of tangible algorithms and design guidelines to promote them in diverse AI contexts. Finally, we invite the HCI and CSCW community to continue exploring tangible algorithms to address the interaction with algorithmic systems, including the collaborative and social computing dynamics they can promote in diverse AI contexts.

Systems involving artificial intelligence (AI) are protagonists in many everyday activities. Moreover, designers are increasingly implementing these systems for groups of users in various social and cooperative domains. Unfortunately, research on personalized recommendation systems often reports negative experiences due to a lack of diversity, control, or transparency. Providing a meta-analysis of the interaction design strategies for group recommendation systems (GRS) offers designers and practitioners a departure to address these issues and imagine new interaction possibilities for this context. Therefore, we systematically reviewed the ACM, IEEE, and Scopus digital libraries to identify GRS interface designs, resulting in a final corpus of 142 academic papers. After a systematic coding process, we used descriptive statistics and thematic analysis to uncover the current state of the art regarding interaction design strategies for GRS in six areas: (1) application domains; (2) devices chosen to implement the systems; (3) prototype fidelity; (4) strategies for profile transparency, justification, control, and diversity; (5) strategies for group formation and final group consensus; and, (6) evaluation methods applied in user studies during the design process. Based on our findings, we present an exhaustive typology of interaction design strategies for GRS and a set of research opportunities to foster human-centered interfaces for personalized recommendations in cooperative and social computing contexts.

Fairness is an important aspect in group recommender systems (GRSs). They must ensure that potentially diverse preferences of all group members are taken into consideration when providing recommendations. Previous work has proposed a number of conflict elicitation and merging techniques to produce preferable recommendations for group members. However, we have yet to understand the influence of user personality on the perception of fairness in GRSs. To examine this gap, we use music recommendation as an example domain. We have developed a web-based group music recommender system using the Spotify API and two simple ranking algorithms: one based on the time the songs were voted by users (time-based) and the other based on a dissimilarity score (dissimilarity-based). A within-subjects experiment was conducted with 45 participants divided into groups of 3 (15 groups). Results showed that openness personality has a negative correlation with the perception that fairness is important in groups.

As the use of AI algorithms keeps rising continuously, so does the need for their transparency and accountability. However, literature often adopts a one-size-fits-all approach for developing explanations when in practice, the type of explanations needed depends on the type of end-user. This research will look at user expertise as a variable to see how different levels of expertise influence the understanding of explanations. The first iteration consists of developing two common types of explanations (visual and textual explanations) that explain predictions made by a general class of predictive model learners. These explanations are then evaluated by users of different expertise backgrounds to compare the understanding and ease-of-use of each type of explanation with respect to the different expertise groups. Results show strong differences between experts and lay users when using visual and textual explanations, as well as lay users having a preference for visual explanations which they perform significantly worse with. To solve this problem, the second iteration of this research focuses on the shortcomings of the first two explanations and tries to minimize the difference in understanding between both expertise groups. This is done through the means of developing and testing a candidate solution in the form of hybrid explanations, which essentially combine both visual and textual explanations. This hybrid form of explanations shows a significant improvement in terms of correct understanding (for lay users in particular) when compared to visual explanations, whilst not compromising on ease-of-use at the same time.

In online learning, teachers need constant feedback about their students’ progress and regulation needs. Learning Analytics Dashboards for process-oriented feedback can be a valuable tool for this purpose. However, few such dashboards have been proposed in literature, and most of them lack empirical validation or grounding in learning theories. We present a teacher-facing dashboard for process-oriented feedback in online learning, co-designed and evaluated through an iterative design process involving teachers and visualization experts. We also reflect on our design process by discussing the challenges, pitfalls, and successful strategies for building this type of dashboard.

TEST 02 - Elsevier's Scopus, the largest abstract and citation database of peer-reviewed literature. Search and access research from the science, technology, medicine, social sciences and arts and humanities fields.

Recommender algorithms play an active role in many everyday activities. However, personalized recommendations often produce negative experiences due to a lack of awareness, control, or transparency. Allowing users to materialize their algorithmic imaginaries exposes how they experience, perceive, and imagine recommender algorithms. Moreover, it can unearth novel and previously unattended design opportunities for tangible interactions with algorithms. Therefore, we explored how 15 users of a famous movie recommender system materialized tangible designs to reflect and discuss their algorithmic imaginaries during co-design workshops and interviews. Using thematic analysis, we identified two forms of algorithmic imaginaries that can inspire tangible interactions with recommender algorithms: metaphoric and datafied representations. Complementary themes exposed the influence of contextual factors and diverse negative attitudes towards personalized movie recommendations. Based on these findings, we suggest design opportunities and suggestions for improving the algorithmic experience of movie recommendations and similar systems through tangible user interfaces.

Information visualization (infovis) is a powerful tool for exploring rich datasets. Within humanistic research, rich qualitative data and domain culture make traditional infovis approaches appear reductive and disconnected, leading to low adoption. In this paper, we use a multi-step approach to scrutinize the relationship between infovis and the humanities and suggest new directions for it. We first look into infovis from the humanistic perspective by exploring the humanistic literature around infovis. We validate and expand those findings though a co-design workshop with humanist and infovis experts. Then, we translate our findings into guidelines for designers and conduct a design critique exercise to explore their effect on the perception of humanist researchers. Based on these steps, we introduce Layers of Meaning, a framework to reduce the semantic distance between humanist researchers and visualizations of their research material, by grounding infovis tools in time and space, physicality, terminology, nuance, and provenance.

Learning analytics dashboards are at the core of the LAK vision to involve the human into the decision-making process. The key focus of these dashboards is to support better human sense-making and decision-making by visualising data about learners to a variety of stakeholders. Early research on learning analytics dashboards focused on the use of visualisation and prediction techniques and demonstrates the rich potential of dashboards in a variety of learning settings. Present research increasingly uses participatory design methods to tailor dashboards to the needs of stakeholders, employs multimodal data acquisition techniques, and starts to research theoretical underpinnings of dashboards. In this paper, we present these past and present research efforts as well as the results of the VISLA19 workshop on "Visual approaches to Learning Analytics" that was held at LAK19 with experts in the domain to identify and articulate common practices and challenges for the domain. Based on an analysis of the results, we present a research agenda to help shape the future of learning analytics dashboards.

We have become increasingly reliant on recommender systems to help us make decisions in our daily live. As such, it is becoming essential to explain to users how these systems reason to enable them to correct system assumptions and to trust the system. The advantages of explaining the recommendation process has been shown by a vast amount of research. Additionally, previous studies showed that personality affects users' attitudes, tastes and information processing. However, it is still unclear whether personality has an impact on the way users process and perceive explanations. In this paper, we report the results of a study that investigated differences between personal characteristics of the perception and the gaze pattern of a music recommender interface in the presence and absence of explanations. We investigated the differences between Need For Cognition, Musical Sophistication and the Big Five personality traits. Results show empirical evidence of the differences between Musical Sophistication and Openness on both perception and gaze pattern. We found that users with a high Musical Sophistication and a low Openness score benefit the most from explanations.

An increasing body of research indicates that transparency in recommender systems affects trust of users. Additionally, a vast amount of studies already showed that personality impacts the way users perceive a recommender system. However, only recently, research has begun to investigate the effects of cognitive style on the perception of recommender systems. Furthermore, it is still unclear whether this cognitive style also affects the interaction strategies of users, and whether the reason why and when users want transparency is affected by this cognitive style. Additionally, despite the ubiquitous presence of recommender systems on mobile environments, no study has investigated the effect of transparency for mobile music recommender systems. In this paper, we report the results of a within-subject study (N=25) on a mobile music recommender system where we investigated the effect of cognitive styles on three different aspects: the interaction strategies with the different applications, the reasons why and when users want transparency and the effect of transparency on the trust of users. The results show that users with a rational thinking style put more effort in seeking the best recommendations and that they want scrutable explanations to adjust the recommendation. In contrast, intuitive thinkers only need explanations when they search for a very specific kind of music.

User beliefs about algorithmic systems are constantly co-produced through user interaction and the complex socio-technical systems that generate recommendations. Identifying these beliefs is crucial because they influence how users interact with recommendation algorithms. With no prior work on user beliefs of algorithmic video recommendations, practitioners lack relevant knowledge to improve the user experience of such systems. To address this problem, we conducted semi-structured interviews with middle-aged YouTube video consumers to analyze their user beliefs about the video recommendation system. Our analysis revealed different factors that users believe influence their recommendations. Based on these factors, we identified four groups of user beliefs: Previous Actions, Social Media, Recommender System, and Company Policy. Additionally, we propose a framework to distinguish the four main actors that users believe influence their video recommendations: the current user, other users, the algorithm, and the organization. This framework provides a new lens to explore design suggestions based on the agency of these four actors. It also exposes a novel aspect previously unexplored: the effect of corporate decisions on the interaction with algorithmic recommendations. While we found that users are aware of the existence of the recommendation system on YouTube, we show that their understanding of this system is limited.

Critiquing-based recommender systems aim to elicit more accurate user preferences from users' feedback toward recommendations. However, systems using a graphical user interface (GUI) limit the way that users can critique the recommendation. With the rise of chatbots in many application domains, they have been regarded as an ideal platform to build critiquing-based recommender systems. Therefore, we present MusicBot, a chatbot for music recommendations, featured with two typical critiquing techniques, user-initiated critiquing (UC) and system-suggested critiquing (SC). By conducting a within-subjects (N=45) study with two typical scenarios of music listening, we compared a system of only having UC with a hybrid critiquing system that combines SC with UC. Furthermore, we analyzed the effects of four personal characteristics,musical sophistication (MS), desire for control (DFC), chatbot experience (CE), and tech savviness (TS), on the user's perception and interaction of the recommendation in MusicBot. In general, compared with UC, SC yields higher perceived diversity and efficiency in looking for songs; combining UC and SC tends to increase user engagement. Both MS and DFC positively influence several key user experience (UX) metrics of MusicBot such as interest matching, perceived controllability, and intent to provide feedback.

Job mediation services can assist job seekers in finding suitable employment through a personalised approach. Consultation or mediation sessions, supported by personal profile data of the job seeker, help job mediators understand personal situation and requests. Prediction and recommendation systems can directly provide job seekers with possible job vacancies. However, incorrect or unrealistic suggestions, and bad interpretations can result in bad decisions or demotivation of the job seeker. This paper explores how an interactive dashboard visualising prediction and recommendation output can help support the dialogue between job mediator and job seeker, by increasing the "explainability" and providing mediators with control over the information that is shown to job seekers.

Recommender systems have been increasingly used in online services that we consume daily, such as Facebook, Netflix, YouTube, and Spotify. However, these systems are often presented to users as a "black box", i.e. the rationale for providing individual recommendations remains unexplained to users. In recent years, various attempts have been made to address this black box issue by providing textual explanations or interactive visualisations that enable users to explore the provenance of recommendations. Among other things, results demonstrated benefits in terms of precision and user satisfaction. Previous research had also indicated that personal characteristics such as domain knowledge, trust propensity and persistence may also play an important role on such perceived benefits. Yet, to date, little is known about the effects of personal characteristics on explaining recommendations. To address this gap, we developed a music recommender system with explanations and conducted an online study using a within-subject design. We captured various personal characteristics of participants and administered both qualitative and quantitative evaluation methods. Results indicate that personal characteristics have significant influence on the interaction and perception of recommender systems, and that this influence changes by adding explanations. For people with a low need for cognition are the explained recommendations the most beneficial. For people with a high need for cognition, we observed that explanations could create a lack of confidence. Based on these results, we present some design implications for explaining recommendations.

The dynamics of the labor market and the tasks with which jobs are being composed are continuously evolving. Job mobility is not evident, and providing effective recommendations in this context has also been found to be particularly challenging. In this paper, we present Labor Market Explorer, an interactive dashboard that enables job seekers to explore the labor market in a personalized way based on their skills and competences. Through a user-centered design process involving job seekers and job mediators, we developed this dashboard to enable job seekers to explore job recommendations and their required competencies, as well as how these competencies map to their profile. Evaluation results indicate the dashboard empowers job seekers to explore, understand, and find relevant vacancies, mostly independent of their background and age.

Music preferences are likely to depend on contextual characteristics such as location and activity. However, most recommender systems do not allow users to adapt recommendations to their current context. We therefore built ContextPlay, a context-aware music recommender that enables user control for both contextual characteristics and music preferences. By conducting a mixed-design study (N=114) with four typical scenarios of music listening, we investigate the effect of controlling contextual characteristics in a music recommender system on four aspects: perceived quality, diversity, effectiveness, and cognitive load. Compared to our baseline which only allows to specify music preferences, having additional control for context leads to higher perceived quality and does not increase cognitive load. We also find that the contexts of mood, weather, and location tend to influence user perception of the system. Moreover, we found that users are more likely to modify contexts and their profile during relaxing activities.

This paper aims to link student facing Learning Analytics Dashboards (LADs) to the corpus of research on Open Learner Models (OLMs), as both have similar goals. We conducted a systematic review of literature on OLMs and compared the results with a previously conducted review of LADs for learners in terms of (i) data use and modelling, (ii) key publication venues, (iii) authors and articles, (iv) key themes, and (v) system evaluation. We highlight the similarities and differences between the research on LADs and OLMs. Our key contribution is a bridge between these two areas as a foundation for building upon the strengths of each. We report the following key results from the review: in reports of new OLMs, almost 60% are based on a single type of data; 33% use behavioral metrics; 39% support input from the user; 37% have complex models; and just 6% involve multiple applications. Key associated themes include intelligent tutoring systems, learning analytics, and self-regulated learning. Notably, compared with LADs, OLM research is more likely to be interactive (81% of papers compared with 31% for LADs), report evaluations (76% versus 59%), use assessment data (100% versus 37%), provide a comparison standard for students (52% versus 38%), but less likely to use behavioral metrics, or resource use data (33% against 75% for LADs). In OLM work, there was a heightened focus on learner control and access to their own data.

Our work focuses on a multi-institutional implementation and evaluation of a Learning Analytics Dashboards (LAD) at scale, providing feedback to N=337 aspiring STEM (science, technology, engineering and mathematics) students participating in a region-wide positioning test before entering the study program. Study advisors were closely involved in the design and evaluation of the dashboard. The multi-institutional context of our case study requires careful consideration of external stakeholders and data ownership and portability issues, which gives shape to the technical design of the LAD. Our approach confirms students as active agents with data ownership, using an anonymous feedback code to access the LAD and to enable students to share their data with institutions at their discretion. Other distinguishing features of the LAD are the support for active content contribution by study advisors and LATEX type-setting of question item feedback to enhance visual recognizability. We present our lessons learnt from a first iteration in production.

This paper presents an evaluation of a learning dashboard that supports the dialogue between a student and a study advisor. The dashboard was designed, developed, and evaluated in collaboration with study advisers. To ensure scalability to other contexts, the dashboard uses data that is commonly available at any higher education institute. It visualizes the grades of the student, an overview of the progress through the year, his/her position in comparison with peers, sliders to plan the next years and a prediction of the length of the bachelor program for this student in years based on historic data. The dashboard was deployed at KU Leuven, Belgium and used in September 2017 to support 224 sessions between students and study advisers. We observed twenty of these conversations. We also collected feedback from 101 students with questionnaires. Results of our observations indicate that the dashboard primarily triggers insights at the beginning of a conversation. The number of insights and the level of these insights (factual, interpretative and reflective) depends on the context of the conversation. Most insights were triggered in conversations with students doubting to continue the program, indicating that our dashboard is useful to support difficult decision-making processes.

Previous research has found that enabling users to control the recommendation process increases user satisfaction. However, providing additional controls also increases cognitive load, and different users have different needs for control. Therefore, in this study, we investigate the effect of two personal characteristics: musical sophistication and visual memory capacity. We designed a visual user interface, on top of a commercial music recommender, with different controls: interactions with recommendations (i.e., the output of a recommender system), the user profile (i.e., the top listened songs), and algorithm parameters (i.e., weights in an algorithm). We created eight experimental settings with combinations of these three user controls and conducted a between-subjects study (N=240), to explore the effect on cognitive load and recommendation acceptance for different personal characteristics. We found that controlling recommendations is the most favorable single control element. In addition, controlling user profile and algorithm parameters was the most beneficial setting with multiple controls. Moreover, the participants with high musical sophistication perceived recommendations to be of higher quality, which in turn lead to higher recommendation acceptance. However, we found no effect of visual working memory on either cognitive load or recommendation acceptance. This work contributes an understanding of how to design control that hits the sweet spot between the perceived quality of recommendations and acceptable cognitive load.

When recommendations become increasingly personalized, users are often presented with a narrower range of content. To mitigate this issue, diversity-enhanced user interfaces for recommender systems have in the past found to be effective in increasing overall user satisfaction with recommendations. However, users may have different requirements for diversity, and consequently different visualization requirements. In this paper, we evaluate two visual user interfaces, SimBub and ComBub, to present the diversity of a music recommender system from different perspectives. SimBub is a baseline bubble chart that shows music genres and popularity by color and size, respectively. In addition, ComBub visualizes selected audio features along the X and Y axis in a more advanced and complex visualization. Our goal is to investigate how individual traits such as musical sophistication (MS) and visual memory (VM) influence the satisfaction of the visualization for perceived music diversity, overall usability, and support to identify blind-spots. We hypothesize that music experts, or people with better visual memory, will perceive higher diversity in ComBub than SimBub. A within-subjects user study (N=83) is conducted to compare these two visualizations. Results of our study show that participants with high MS and VM tend to perceive significantly higher diversity from ComBub compared to SimBub. In contrast, participants with low MS perceived significantly higher diversity from SimBub than ComBub; however, no significant result is found for the participants with low VM. Our research findings show the necessity of considering individual traits while designing diversity-aware interfaces.

The "black box'' nature of today's recommender systems raises a number of challenges for users, including a lack of trust and limited user control. Providing more user control is interesting to enable end-users to help steer the recommendation process with additional input and feedback. However, different users may have different preferences with regard to such control. To the best of our knowledge, no research has investigated the effect of personal characteristics on visual control techniques in the music recommendation domain. In this paper, we present results of a user study on the web using two different visualisation techniques (a radar chart and sliders) that allows users to control Spotify recommendations. A within-subject design withLatin Square counterbalancing measures was used for the study. Results indicate that the radar chart helped the participants discover a significantly higher number of new songs compared to the sliders. We also found that users' experience with Spotify had an influence on their interaction with different musical attributes. The participants who used Spotify frequently and users with a high individual musical sophistication interacted with the attributes significantly more with the radar chart compared to the sliders. Individual musical sophistication also had a significant impact on their interaction with the interaction techniques. The participants with high musical sophistication interacted significantly more with the radar chart in comparison to the sliders. Based on the feedback from our participants, we provide design suggestions to further improve user control in music recommendation.

We conducted a literature review on systems that track learning analytics data (e.g., resource use, time spent, assessment data, etc.) and provide a report back to students in the form of visualizations, feedback, or recommendations. This review included a rigorous article search process; 945 articles were identified in the initial search. After filtering out articles that did not meet the inclusion criteria, 94 articles were included in the final analysis. Articles were coded on five categories chosen based on previous work done in this area: functionality, data sources, design analysis, perceived effects, and actual effects. The purpose of this review is to identify trends in the current student-facing learning analytics reporting system literature and provide recommendations for learning analytics researchers and practitioners for future work.

Targeted advertising reaches users based on various traits, such as demographics or behaviour. However, users are often reluctant to accept ads. We hypothesise that users are more open to targeted advertising if they can inspect, control and thereby understand the process of ad selection. We conducted a between-subjects study (N=200) to investigate to what extent four key aspects of ads (Quality, Behavioural Intention, Understanding and Attitude) may be affected by transparency and user control using a flow chart. Our results indicate that positive effects of flow charts reported from other domains may also be applicable to advertising: Using flow charts to provide transparency together with user control is found to have more positive effects on domain-specific quality measures than established, text-based approaches and using either of the techniques in isolation. The paper concludes with recommendations for practitioners aiming to improve user response to ads.

How can you discover something new, that matches your interest? Recommender Systems have been studied since the 90ies. Their benefit comes from guiding a user through the density of the information jungle to useful knowledge clearings. Early research on recommender systems focuses on algorithms and their evaluation to improve recommendation accuracy using F-measures and other methodologies from signal-detection theory. Present research includes other aspects such as human factors that affect the user experience and interactive visualization techniques to support transparency of results and user control. In this paper, we analyze all publications on recommender systems from the scopus database, and particularly also papers with such an HCI focus. Based on an analysis of these papers, future topics for recommender systems research are identified, which include more advanced support for user control, adaptive interfaces, affective computing and applications in high risk domains.

Several approaches have been researched to help people deal with abundance of information. An important feature pioneered by social tagging systems and later used in other kinds of social systems is the ability to explore different community relevance prospects by examining items bookmarked by a specific user or items associated by various users with a specific tag . A ranked list of recommended items offered by a specific recommender engine can be considered as another relevance prospect. The problem that we address is that existing personalized social systems do not allow their users to explore and combine multiple relevance prospects. Only one prospect can be explored at any given time—a list of recommended items, a list of items bookmarked by a specific user, or a list of items marked with a specific tag. In this article, we explore the notion of combining multiple relevance prospects as a way to increase effectiveness and trust. We used a visual approach to recommend articles at a conference by explicitly presenting multiple dimensions of relevance. Suggestions offered by different recommendation techniques were embodied as recommender agents to put them on the same ground as users and tags. The results of two user studies performed at academic conferences allowed us to obtain interesting insights to enhance user interfaces of personalized social systems. More specifically, effectiveness and probability of item selection increase when users are able to explore and interrelate prospects of items relevance—that is, items bookmarked by users, recommendations and tags. Nevertheless, a less-technical audience may require guidance to understand the rationale of such intersections.

In this paper, we briefly describe the goal and activities of the LAK15 workshop on Visual Aspects of Learning analytics.

Research on recommender systems has traditionally focused on the development of algorithms to improve accuracy of recommendations. So far, little research has been done to enable user interaction with such systems as a basis to support exploration and control by end users. In this paper, we present our research on the use of information visualization techniques to interact with recommender systems. We investigated how information visualization can improve user understanding of the typically black-box rationale behind recommendations in order to increase their perceived relevance and meaning and to support exploration and user involvement in the recommendation process. Our study has been performed using TalkExplorer, an interactive visualization tool developed for attendees of academic conferences. The results of user studies performed at two conferences allowed us to obtain interesting insights to enhance user interfaces that integrate recommendation technology. More specifically, effectiveness and probability of item selection both increase when users are able to explore and interrelate multiple entities -- i.e. items bookmarked by users, recommendations and tags.

This paper reports on our research on the use of learning analytics dashboards to support awareness, self-reflection, sensemaking and impact for learners. So far, little research has been done to evaluate such dashboards with students and to assess their impact on learning. In this paper, we present the results of an evaluation study of our dashboard, called StepUp!, and the extent to which it addresses issues and needs of our students. Through brainstorming sessions with our students, we identified and prioritized learning issues and needs. In a second step, we deployed StepUp! during one month and we evaluated to which extent our dashboard addresses the issues and needs identified earlier in different courses. The results show that our tool has potentially higher impact for students working in groups and sharing a topic than students working individually on different topics.

Learning Analytics, an emerging field concerned with analyzing the vast data "given off" by learners in technology supported settings to inform educational theory and practice, has from its inception taken a multidisciplinary approach that integrates studies of learning with technological capabilities. In this introduction to the Proceedings of the Third International Learning Analytics & Knowledge Conference, we discuss how Learning Analytics must function in the "middle space" where learning and analytic concerns meet. Dialogue in this middle space involves diverse stakeholders from multiple disciplines with various conceptions of the agency and nature of learning. We hold that a singularly unified field is not possible nor even desirable if we are to leverage the potential of this diversity, but progress is possible if we support "productive multivocality" between the diverse voices involved, facilitated by appropriate use of boundary objects. We summarize the submitted papers and contents of these Proceedings to characterize the voices and topics involved in the multivocal discourse of Learning Analytics.

The main objective of the 1st International Workshop on Learning Analytics and Linked Data (#LALD2012) is to connect the research efforts on Linked Data and Learning Analytics in order to create visionary ideas and foster synergies between the two young research fields. Therefore, the workshop will collect, explore, and present datasets, technologies and applications for Technology Enhanced Learning (TEL) to discuss Learning Analytics approaches that make use of educational data or Linked Data sources. During the workshop, an overview of available educational datasets and related initiatives will be given. The participants will have the opportunity to present their own research with respect to educational datasets, technologies and applications and discuss major challenges to collect, reuse, and share these datasets.

Increasing motivation of students and helping them to reflect on their learning processes is an important driver for learning analytics research. This paper presents our research on the development of a dashboard that enables self-reflection on activities and comparison with peers. We describe evaluation results of four iterations of a design based research methodology that assess the usability, use and usefulness of different visualizations. Lessons learned from the different evaluations performed during each iteration are described. In addition, these evaluations illustrate that the dashboard is a useful tool for students. However, further research is needed to assess the impact on the learning process.

The Recommender System Challenge 2012 invited participants to work on two tracks with real-world datasets and to submit their contributions that would be related to specific problem contexts. First of all, it asked participants to develop new algorithms and to compare them to other algorithms in given settings; in addition, it asked participants to explore with new recommendation methods, services, as well as added-value services related to recommendation.

Retrieving and reusing Learning Objects can lighten the workload of constructing new on-line courses or Technology Supported Learning Systems. The paper presents ErauzOnt, a framework for the automatic generation of new Learning Objects from electronic documents using domain ontologies and Natural Language Processing techniques.

In the world of recommender systems, it is a common practice to use public available datasets from different application environments (e.g. MovieLens, Book-Crossing, or Each-Movie) in order to evaluate recommendation algorithms. These datasets are used as benchmarks to develop new recommendation algorithms and to compare them to other algorithms in given settings. In this paper, we explore datasets that capture learner interactions with tools and resources. We use the datasets to evaluate and compare the performance of different recommendation algorithms for learning. We present an experimental comparison of the accuracy of several collaborative filtering algorithms applied to these TEL datasets and elaborate on implicit relevance data, such as downloads and tags, that can be used to improve the performance of recommendation algorithms.

This workshop presents the current status related to the design, development and evaluation of recommender systems in educational settings. It emphasizes the importance of recommender systems for Technology Enhanced Learning (TEL) to support learners with personalized learning resources and suitable peer learners to improve their learning process. Moreover, it proposes a dataTEL challenge to obtain data sets from TEL applications that can be used to benchmark algorithms specifically for the TEL context.