Recent advances in artificial intelligence (AI), including generative models and socially interactive agents, are reshaping the design of interactive systems and raising new challenges for Human–Computer Interaction (HCI). While AI research has traditionally focused on improving performance and scalability, HCI emphasizes usability, transparency, and the broader social implications of technology, all aspects that imply the application of proper human-centered design methods. Bridging these perspectives is increasingly critical as AI moves from backend functionality to a central role in user interaction. This paper reports on the workshop Future Horizons in Human–AI Interaction: Joint Perspectives from HCI and AI, held at AVI 2026. It synthesizes key insights on emerging challenges and opportunities in designing human-centered AI systems, emphasizing the importance of integrating perspectives from HCI and AI, to advance the design of interactive AI systems that are both technically robust and aligned with human values.

Providing athletes with feedback to refine their technique is key in sports coaching and is critical for improving performance and preventing injuries. However, access to expert coaching is often limited. In this paper, we explore a novel counterfactual-based feedback system as a complementary tool to expert coaching and conduct a small-scale user study to explore its perceived usability. Our approach uses an augmented GANterfactual framework, a modified CycleGAN architecture with a classifier-guided counterfactual loss, to synthesize plausible, actionable feedback. As a test bed for our approach, we use the complex motor task of javelin throwing, a sport that is characterized by high biomechanical demands and injury risk. As we are interested in the perceived usability of our approach, we conduct a user study with 21 sports students. The subjective feedback provided by participants of our user study shows that, while pose-based counterfactual feedback visualizations are appreciated by athletes, for some users they require too much domain-specific knowledge and are not “coach-like” enough. We find that athletes are looking for accompanying textual feedback, supporting recent research in the field of feedback generation for sports and motor learning.

The increasing pervasiveness of software-intensive systems requires involving domain experts more directly in technological development. Visual models, expressed in semi-formal notations, can act as shared artefacts that support communication and collaboration between developers and domain experts. However, modelling with semi-formal notations can be challenging for novice modellers. This study presents an AI-infused, web-based modelling tool designed to support users in formalising domain knowledge without requiring advanced modelling skills. The tool features a block-based, domain-specific language that automatically transforms user-generated structures into semi-formal diagrams. AI-based functionalities include a diagram reader, contextual hints, natural-language instructions, and interaction logging. We evaluated the tool through a Wizard of Oz experiment with agronomists in digital agriculture, where participants completed an exploratory modelling task while interacting with AI assistance. Results reveal three key design implications: (i) adaptive AI support accommodating diverse modelling strategies, (ii) concise, actionable guidance delivered at moments of difficulty, and (iii) practice-oriented assistance that preserves user agency and supports learning-by-doing.

Recommender systems are increasingly used in mobile health interventions, such as managing Chronic Musculoskeletal Pain (CMP). While researchers have highlighted the importance of explaining health-related recommendations to lay users, with benefits such as increased trust and a higher tendency to follow up on these recommendations, how to design explanations for lay users in critical contexts such as health remains largely unexplored. To address this gap, we develop a mobile health application to support users with CMP through coaching and personalised health recommendations delivered via a conversational rule-based recommender system. This article describes the three-phase iterative development of the RS, involving health experts and end users. In the first iteration, we conduct a preliminary validation study with \(N=282\) participants to ensure the app’s validity and improve the initial set of health recommendations. Next, two user studies are conducted centred around designing effective and understandable explanations for these recommendations. First, we design six explanation modalities tailored towards lay users, and through a qualitative study ( \(N=11\) ), extract initial design guidelines for explaining health recommendations, finding a strong preference towards feature importance explanations and identifying issues with modalities that highlight negative emotions. Given these results, we explore whether extending feature importance explanations with textual information into a ‘hybrid’ explanation could benefit end users, and whether these benefits depend on a user’s personal characteristics (need for cognition and ease-of-satisfaction). Through a mixed-methods study with \(N=262\) participants, we find that the hybrid modality significantly increased user trust, transparency, persuasiveness, usefulness and satisfaction compared to unimodal explanations. However, users with a higher need for cognition rate unimodal explanations more positively than hybrid ones.

Abstract Research shows that user traits can modulate the use of visualization systems and have a measurable influence on users' accuracy, speed, and attention when performing visual analysis. This highlights the importance of user‐adaptive visualization that can modify themselves to the characteristics and preferences of the user. However, there are very few such visualization systems, as creating them requires broad knowledge from various sub‐domains of the visualization community. A user‐adaptive system must consider which user traits they adapt to, their adaptation logic and the types of interventions they support. In this STAR, we survey a broad space of existing literature and consolidate them to structure the process of creating user‐adaptive visualizations into five components: Capture Ⓐ Input from the user and any relevant peripheral information. Perform computational Ⓑ User Modelling with this input to construct a Ⓒ User Representation . Employ Ⓓ Adaptation Assignment logic to identify when and how to introduce Ⓔ Interventions . Our novel taxonomy provides a road map for work in this area, describing the rich space of current approaches and highlighting open areas for future work.

Semi-structured interviews are a critical qualitative method in many areas, including CSCW and HCI, enabling researchers to uncover deep contextualized insights. Using this flexible method, interviewers must adapt to interviewees' responses while adhering to the protocol, necessitating strong active listening and real-time analytical skills. While recent studies have explored how AI can support researchers in qualitative analysis, to our knowledge, no research has investigated AI's role in supporting semi-structured interviews while they are underway. Taking one step toward filling this gap, we interviewed 16 researchers with a range of prior interviewing expertise. Our inductive thematic analysis reveals that interviewers expect real-time AI assistance to support research objectives and facilitate interpersonal communication, but also have concerns about its impact on long-term skill development. We discuss how semi-structured interviews differ from other problem-solving or creative human-AI collaboration contexts, highlighting the time constraints, multimodal collaboration, and the triangular dynamic among interviewers, interviewees, and AI. We also delve into how interviewers' levels of expertise affect their envisioned interviewer-AI collaboration. We then propose design challenges for future CSCW work on AI-driven assistants in interview contexts.

In recent years, the increasing availability of personal data has raised concerns regarding privacy and security. One of the critical processes to address these concerns is data anonymization, which aims to protect individual privacy and prevent the release of sensitive information. This research focuses on the importance of face anonymization. Therefore, we introduce GANonymization, a novel face anonymization framework with facial expression-preserving abilities. Our approach is based on a high-level representation of a face, which is synthesized into an anonymized version based on a generative adversarial network (GAN). The effectiveness of the approach was assessed by evaluating its performance in removing identifiable facial attributes to increase the anonymity of the given individual face. Additionally, the performance of preserving facial expressions was evaluated on several affect recognition datasets and outperformed the state-of-the-art methods in most categories. Finally, our approach was analyzed for its ability to remove various facial traits, such as jewelry, hair color, and multiple others. Here, it demonstrated reliable performance in removing these attributes. Our results suggest that GANonymization is a promising approach for anonymizing faces while preserving facial expressions.

The HumanEYEze 2024 workshop aims to explore the role of eye tracking in developing human-centered multimodal AI systems. Over the past two decades, eye tracking has evolved from a diagnostic tool to an important input modality for real-time interactive systems, driven by advancements in hardware that have improved its affordability, availability, and performance. Initially used in specialized applications, eye tracking now significantly impacts research on gaze-based multimodal interaction. Recently, eye-based user and context modeling has emerged, utilizing eye movements to provide rich insights into user behavior and interaction contexts. The workshop aims to bring together researchers from eye tracking, multimodal human-computer interaction, and AI. It aims to enhance understanding of integrating eye tracking into multimodal human-centered computing. The expected outcomes include fostering collaborations and promoting knowledge exchange.

A natural way to resolve different points of view and form opinions is through exchanging arguments and knowledge. Facing the vast amount of available information on the internet, people tend to focus on information consistent with their beliefs. To support a fair and unbiased opinion-building process, we propose an intelligent agent in the form of a chatbot that engages in a deliberative dialogue with a human. In contrast to persuasive systems, the chatbot aims to provide a diverse and representative overview - embedded in a conversation with the user. To account for a reflective and unbiased exploration of the topic, we enable the system to intervene if the user is too focused on their pre-existing opinion. To achieve that, the agent employs a metric to assess the user’s focus on challenger arguments.

Adaptive support can help learners in Open-Ended Learning Environments (OELEs), where the free-form nature of the interaction can be confusing to students. In this paper, we design and evaluate an Intelligent Pedagogical Agent (IPA) for an OELE designed to foster Computational Thinking (CT). Specifically, we design help interventions for an in-the-wild scenario where students interact with the OELE in an unmonitored, self-directed manner. We build a student model by extracting meaningful student behaviors on real-world interaction data obtained during interaction in online classrooms and including expert insights. We show that these student models perform better than a baseline and have the potential for adaptive support in self-directed interaction with the OELE. We design an IPA with the help of teachers, leveraging the student behaviors extracted from data. Lastly, we get insights into the value of these help interventions by empirically evaluating the IPA in a formal user study.

Difficulty adjustment in practice exercises has been shown to be beneficial for learning. However, previous research has mostly investigated close-ended tasks, which do not offer the students multiple ways to reach a valid solution. Contrary to this, in order to learn in an open-ended learning task, students need to effectively explore the solution space as there are multiple ways to reach a solution. For this reason, the effects of difficulty adjustment could be different for open-ended tasks. To investigate this, as our first contribution, we compare different methods of difficulty adjustment in a user study conducted with 86 participants. Furthermore, as the practice behavior of the students is expected to influence how well the students learn, we additionally look at their practice behavior as a post-hoc analysis. Therefore, as a second contribution, we identify different types of practice behavior and how they link to students’ learning outcomes and subjective evaluation measures as well as explore the influence the difficulty adjustment methods have on the practice behaviors. Our results suggest the usefulness of taking into account the practice behavior in addition to only using the practice performance to inform adaptive intervention and difficulty adjustment methods.

Previous research on an Intelligent Tutoring System (referred to as ACSP), showed the need to personalize explanations of its AI-driven hints for users with low Need for Cognition (N4C) and low Conscientiousness (Cons.). Specifically, this work found that explanations should be provided to these users with the objective of increasing user interaction with them. In this paper, we present and evaluate design alterations to the original ACSP explanation interface aimed at achieving this objective. Our results provide initial evidence that the implemented personalization, in the form of the design alterations, had a positive impact on users with low N4C and Cons., by increasing attention to explanations and contributing to learning gains.

EXplainable Artificial Intelligence (XAI) is a vibrant research topic in the artificial intelligence community. It is raising growing interest across methods and domains, especially those involving high stake decision-making, such as the biomedical sector. Much has been written about the subject, yet XAI still lacks shared terminology and a framework capable of providing structural soundness to explanations. In our work, we address these issues by proposing a novel definition of explanation that synthesizes what can be found in the literature. We recognize that explanations are not atomic but the combination of evidence stemming from the model and its input-output mapping, and the human interpretation of this evidence. Furthermore, we fit explanations into the properties of faithfulness (i.e., the explanation is an accurate description of the model’s inner workings and decision-making process) and plausibility (i.e., how much the explanation seems convincing to the user). Our theoretical framework simplifies how these properties are operationalized, and it provides new insights into common explanation methods that we analyze as case studies. We also discuss the impact that our framework could have in biomedicine, a very sensitive application domain where XAI can have a central role in generating trust.

We explore eXplainable AI (XAI) to enhance user experience and understand the value of explanations in AI-driven pedagogical decisions within an Intelligent Pedagogical Agent (IPA). Our real-time and personalized explanations cater to students' attitudes to promote learning. In our empirical study, we evaluate the effectiveness of personalized explanations by comparing three versions of the IPA: (1) personalized explanations and suggestions, (2) suggestions but no explanations, and (3) no suggestions. Our results show the IPA with personalized explanations significantly improves students' learning outcomes compared to the other versions.

Learning can be improved by providing personalized feedback adapting to the emotions that the learner may be experiencing. There is initial evidence that co-occurring emotions can be predicted during learning in Intelligent Tutoring Systems (ITS) through eye-tracking and interaction data. Predicting co-occurring emotions is a complex task and merging datasets has the potential to improve predictive performance. In this paper, we combine data from two user studies with an ITS, and analyze whether there is an improvement in predictive performance of co-occurring emotions, despite the user studies using different eye-trackers. In the pursuit towards developing real affect-aware ITS, we look at whether we can isolate classifiers that perform better than a baseline. In this regard we perform a series of statistical analyses and test out the predictive performance of standard machine learning models as well as an ensemble classifier for the task of predicting co-occurring emotions.

There has been increasing interest in non-invasive predictors of Alzheimer's disease (AD) as an initial screen for this condition. Previously, successful attempts leveraged eye-tracking and language data generated during picture narration and reading tasks. These results were obtained with high-end, expensive eye-trackers. Instead, we explore classification using eye-tracking data collected with a webcam, where our classifiers are built using a deep-learning approach. Our results show that the webcam gaze classifier is not as good as the classifier based on high-end eye-tracking data. However, the webcam-based classifier still beats the majority-class baseline classifier in terms of AU-ROC, indicating that predictive signals can be extracted from webcam gaze tracking. Hence, although our results indicate that there is still a long way to go before webcam gaze tracking can reach practical relevance, they still provide an encouraging proof of concept that this technology should be further explored as an affordable alternative to high-end eye-trackers for the detection of AD.

Research has shown that free-form Game-Design (GD) environments can be very effective in fostering Computational Thinking (CT) skills at a young age. However, some students can still need some guidance during the learning process due to the highly open-ended nature of these environments. Intelligent Pedagogical Agents (IPAs) can be used to provide personalized assistance in real-time to alleviate this challenge. This paper presents our results in evaluating such an agent deployed in a real-word free-form GD learning environment to foster CT in the early K-12 education, Unity-CT. We focus on the effect of repetition by comparing student behaviors between no intervention, 1-shot, and repeated intervention groups for two different errors that are known to be challenging in the online lessons of Unity-CT. Our findings showed that the agent was perceived very positively by the students and the repeated intervention showed promising results in terms of helping students make fewer errors and more correct behaviors, albeit only for one of the two target errors. Building from these results, we provide insights on how to provide IPA interventions in free-form GD environments.

Free-form Game-Design (GD) environments show promise in fostering Computational Thinking (CT) skills at a young age. However, such environments can be challenging to some students due to their highly open-ended nature. Our long-term goal is to alleviate this difficulty via pedagogical agents that can monitor the student interaction with the environment, detect when the student needs help and provide personalized support accordingly. In this paper, we present a preliminary evaluation of one such agent deployed in a real-word free-form GD learning environment to foster CT in the early K-12 education, Unity-CT. We focus on the effect of repetition by comparing student behaviors between no intervention, 1-shot, and repeated intervention groups for two different errors that are known to be challenging in the online lessons of Unity-CT environment. Our findings showed that the agent was perceived very positively by the students and the repeated intervention showed promising results in terms of helping students make less errors and more correct behaviors, albeit only for one of the two target errors. Based on these results, we provide insights on how to improve the delivery of the agent’s interventions in free-form GD environments.

Game-Design (GD) environments show promise in fostering Computational Thinking (CT) skills at a young age. However, such environments can be challenging to some students due to their highly open-ended nature. We propose to alleviate this difficulty by learning interpretable student models from data that can drive personalization of a real-world GD learning environment to the student’s needs. We apply our approach on a dataset collected in ecological settings and evaluate the ability of the generated student models at predicting ineffective learning behaviors over the course of the interaction. We then discuss how these behaviors can be used to define personalized support in GD learning activities, by conducting extensive interviews with experienced instructors.

We study the effectiveness of adaptive interventions at helping users process textual documents with embedded visualizations, a form of multimodal documents known as Magazine-Style Narrative Visualizations (MSNVs). The interventions are meant to dynamically highlight in the visualization the datapoints that are described in the textual sentence currently being read by the user, as captured by eye-tracking. These interventions were previously evaluated in two user studies that involved 98 participants reading excerpts of real-world MSNVs during a 1-hour session. Participants’ outcomes included their subjective feedback about the guidance, and well as their reading time and score on a set of comprehension questions. Results showed that the interventions can increase comprehension of the MSNV excerpts for users with lower levels of a cognitive skill known as visualization literacy. In this article, we aim to further investigate this result by leveraging eye-tracking to analyze in depth how the participants processed the interventions depending on their levels of visualization literacy. We first analyzed summative gaze metrics that capture how users process and integrate the key components of the narrative visualizations. Second, we mined the salient patterns in the users’ scanpaths to contextualize how users sequentially process these components. Results indicate that the interventions succeed in guiding attention to salient components of the narrative visualizations, especially by generating more transitions between key components of the visualization (i.e., datapoints, labels, and legend), as well as between the two modalities (text and visualization). We also show that the interventions help users with lower levels of visualization literacy to better map datapoints to the legend, which likely contributed to their improved comprehension of the documents. These findings shed light on how adaptive interventions help users with different levels of visualization literacy, informing the design of personalized narrative visualizations.

In this article, we investigate the value of gaze-driven adaptive interventions to support the processing of textual documents with embedded visualizations, i.e., Magazine Style Narrative Visualizations (MSNVs). These interventions are provided dynamically by highlighting relevant data points in the visualization when the user reads related sentences in the MSNV text, as detected by an eye-tracker. We conducted a user study during which participants read a set of MSNVs with our interventions, and compared their performance and experience with participants who received no interventions. Our work extends previous findings by showing that dynamic, gaze-driven interventions can be delivered based on reading behaviors in MSNVs, a widespread form of documents that have never been considered for gaze-driven adaptation so far. Next, we found that the interventions significantly improved the performance of users with low levels of visualization literacy, i.e., those users who need help the most due to their lower ability to process and understand data visualizations. However, high literacy users were not impacted by the interventions, providing initial evidence that gaze-driven interventions can be further improved by personalizing them to the levels of visualization literacy of their users.

We leverage eye-tracking data to predict user performance and levels of cognitive abilities while reading magazine-style narrative visualizations (MSNV), a widespread form of multimodal documents that combine text and visualizations. Such predictions are motivated by recent interest in devising user-adaptive MSNVs that can dynamically adapt to a user's needs. Our results provide evidence for the feasibility of real-time user modeling in MSNV, as we are the first to consider eye tracking data for predicting task comprehension and cognitive abilities while processing multimodal documents. We follow with a discussion on the implications to the design of personalized MSNVs.

Encouraged by the success of deep learning in a variety of domains, we investigate the effectiveness of a novel application of such methods for detecting user confusion with eye-tracking data. We introduce an architecture that uses RNN and CNN sub-models in parallel, to take advantage of the temporal and visuospatial aspects of our data. Experiments with a dataset of user interactions with the ValueChart visualization tool show that our model outperforms an existing model based on a Random Forest classifier, resulting in a 22% improvement in combined confused & not confused class accuracies.

We study the effectiveness of adaptive guidance at helping users process textual documents with embedded visualizations, known as narrative visualizations. We do so by leveraging eye tracking to analyze in depth the effect that adaptations meant to guide the user's gaze to relevant parts of the visualizations has on users with different levels of visualization literacy. Results indicate that the adaptations succeed in guiding attention to salient components of the narrative visualizations, especially by generating more transitions between key components of the visualization (i.e., datapoints, labels and legend). We also show that the adaptation helps users with lower levels of visualization literacy to better map datapoints to the legend, which leads in part to improved comprehension of the visualization. These findings shed light on how adaptive guidance helps users with different levels of visualization literacy, informing the design of personalized narrative visualizations.

There is increasing evidence that user characteristics can have a significant impact on visualization effectiveness, suggesting that visualizations could be enriched with personalization mechanisms that better fit each user's specific needs and abilities. In this paper, we contribute to this body of work with a study that investigates the impact of six user characteristics on the effectiveness of time series visualizations, which was not previously investigated in relation to personalizing Information visualization. We report on a controlled user study that compare four possible time series visualization techniques. User performance and how it was affected by user characteristics was measured while performing tasks from a formal taxonomy using Twitter data about real-world events. Our results show that both the personality trait of locus of control and the cognitive ability of verbal working memory influence which visualization is more effective when dealing with demanding and complex tasks. These findings extend the need for personalization to visualizations for time series data, and we discuss them in the context of creating systems that can utilize knowledge of the user's specific characteristics in order to present the most suitable visualization for each user.

Eye-tracking has been extensively used both in psychology for understanding various aspects of human cognition, as well as in human computer interaction (HCI) for evaluation of interface design or as a form of direct input. In recent years, eye-tracking has also been investigated as a source of information for machine learning models that predict relevant user states and traits (e.g., attention, confusion, learning, perceptual abilities). These predictions can then be leveraged by AI agents to model their users and personalize the interaction accordingly. In this talk, Dr. Conati will provide an overview of the research her lab has done in this area, including detecting and modeling user cognitive skills, and affective states, with applications to user-adaptive visualizations, intelligent tutoring systems and health.

We have become increasingly reliant on recommender systems to help us make decisions in our daily live. As such, it is becoming essential to explain to users how these systems reason to enable them to correct system assumptions and to trust the system. The advantages of explaining the recommendation process has been shown by a vast amount of research. Additionally, previous studies showed that personality affects users' attitudes, tastes and information processing. However, it is still unclear whether personality has an impact on the way users process and perceive explanations. In this paper, we report the results of a study that investigated differences between personal characteristics of the perception and the gaze pattern of a music recommender interface in the presence and absence of explanations. We investigated the differences between Need For Cognition, Musical Sophistication and the Big Five personality traits. Results show empirical evidence of the differences between Musical Sophistication and Openness on both perception and gaze pattern. We found that users with a high Musical Sophistication and a low Openness score benefit the most from explanations.

Previous work has shown that some user cognitive abilities relevant for processing information visualizations can be predicted from eye-tracking data. Performing this type of user modeling is important for devising visualizations that can detect a user's abilities and adapt accordingly during the interaction. In this article, we extend previous user modeling work by investigating for the first time interaction data as an alternative source to predict cognitive abilities during visualization processing when it is not feasible to collect eye-tracking data. We present an extensive comparison of user models based solely on eye-tracking data, on interaction data, as well as on a combination of the two. Although we found that eye-tracking data generate the most accurate predictions, results show that interaction data can still outperform a majority-class baseline, meaning that adaptation for interactive visualizations could be enabled even when it is not feasible to perform eye tracking, using solely interaction data. Furthermore, we found that interaction data can predict several cognitive abilities with better accuracy at the very beginning of the task than eye-tracking data, which are valuable for delivering adaptation early in the task. We also extend previous work by examining the value of multimodal classifiers combining interaction data and eye-tracking data, with promising results for some of our target user cognitive abilities. Next, we contribute to previous work by extending the type of visualizations considered and the set of cognitive abilities that can be predicted from either eye-tracking data and interaction data. Finally, we evaluate how noise in gaze data impacts prediction accuracy and find that retaining rather noisy gaze datapoints can yield equal or even better predictions than discarding them, a novel and important contribution for devising adaptive visualizations in real settings where eye-tracking data are typically noisier than in laboratory settings.

We present an experimenter platform for designing and evaluating user-adaptive support in information visualizations. Specifically, this platform leverages eye-tracking data in real time to deliver adaptive support in visualizations based on the user's attentional patterns and individual needs. We describe the main functionalities of this platform, and show an application to support processing of textual documents with embedded bar charts, by dynamically providing highlighting in the charts to guide a user's attention to the relevant information.

Although there is extensive evidence that personalization of interactive systems can improve the user's experience and satisfaction, it is also known that the two main approaches to deliver personalization, namely via customization or system-driven adaptation, have limitations. In particular, many users do not use customize mechanisms, while adaptation can be perceived as intrusive and opaque. In this paper, we explore an intermediary approach to personalization, namely delivering system-driven support to customization. To this end, we study a customization mechanism allowing to choose the type and amount of information displayed by means of information visualizations in a system for decision making, and examine the impact of user differences on the effectiveness of this mechanism. Our results show that, for the users who did use the customization mechanism, customization effectiveness was impacted by their levels of visualization literacy and locus of control. These results suggest that the customization mechanism could be improved by system-driven assistance to customize depending on the user's level of visualization literacy and locus of control.

Recommender systems have been increasingly used in online services that we consume daily, such as Facebook, Netflix, YouTube, and Spotify. However, these systems are often presented to users as a "black box", i.e. the rationale for providing individual recommendations remains unexplained to users. In recent years, various attempts have been made to address this black box issue by providing textual explanations or interactive visualisations that enable users to explore the provenance of recommendations. Among other things, results demonstrated benefits in terms of precision and user satisfaction. Previous research had also indicated that personal characteristics such as domain knowledge, trust propensity and persistence may also play an important role on such perceived benefits. Yet, to date, little is known about the effects of personal characteristics on explaining recommendations. To address this gap, we developed a music recommender system with explanations and conducted an online study using a within-subject design. We captured various personal characteristics of participants and administered both qualitative and quantitative evaluation methods. Results indicate that personal characteristics have significant influence on the interaction and perception of recommender systems, and that this influence changes by adding explanations. For people with a low need for cognition are the explained recommendations the most beneficial. For people with a high need for cognition, we observed that explanations could create a lack of confidence. Based on these results, we present some design implications for explaining recommendations.

In this paper, we present research to uncover how the level of reading comprehension abilities impacts how users process textual documents in English with embedded visualizations (i.e., Magazine Style Narrative Visualizations or MSNVs). We analyze performance and gaze data of users processing MSNVs from two user studies, one run in Canada and one in a non-English speaking European country. Our findings provide important insights toward developing automatic, real-time support to MSNV processing personalized according to users' English reading comprehension abilities.

In this paper we present results from an exploratory user study to uncover which user characteristics (e.g., perceptual speed, verbal working memory, etc.) play a role in how users process textual documents with embedded visualizations (i.e., Magazine Style Narrative Visualizations). We present our findings as a step toward developing user-adaptive support, and provide suggestions on how our results can be leveraged for creating a set of meaningful interventions for future evaluation.

Data science is now impacting the education sector, with a growing number of commercial products and research prototypes providing learning dashboards. From a human-centred computing perspective, the end-user's interpretation of these visualisations is a critical challenge to design for, with empirical evidence already showing that `usable' visualisations are not necessarily effective from a learning perspective. Since an educator's interpretation of visualised data is essentially the construction of a narrative about student progress, we draw on the growing body of work on Data Storytelling (DS) as the inspiration for a set of enhancements that could be applied to data visualisations to improve their communicative power. We present a pilot study that explores the effectiveness of these DS elements based on educators' responses to paper prototypes. The dual purpose is understanding the contribution of each visual element for data storytelling, and the effectiveness of the enhancements when combined.

Previous work has shown that some user cognitive abilities relevant for processing information visualizations can be predicted from eye tracking data. Performing this type of user modeling is important for devising user-adaptive visualizations that can adapt to a user’s abilities as needed during the interaction. In this paper, we contribute to previous work by extending the type of visualizations considered and the set of cognitive abilities that can be predicted from gaze data, thus providing evidence on the generality of these findings. We also evaluate how quality of gaze data impacts prediction.

In this paper we investigate using a variety of behavioral measures collectible with an eye tracker to predict a user's skill acquisition phase while performing various information visualization tasks with bar graphs. Our long term goal is to use this information in real-time to create user-adaptive visualizations that can provide personalized support to facilitate visualization processing based on the user's predicted skill level. We show that leveraging two additional content-independent data sources, namely information on a user's pupil dilation and head distance to the screen, yields a significant improvement for predictive accuracies of skill acquisition compared to predictions made using content-dependent information related to user eye gaze attention patterns, as was done in previous work. We show that including features from both pupil dilation and head distance to the screen improve the ability to predict users' skill acquisition state, beating both the baseline and a model using only content-dependent gaze information.

There is increasing evidence that the effectiveness of Information Visualization (Infovis) is affected by the user needs and abilities. For instance, cognitive abilities (e.g., perceptual speed, working memory) [e.g., 1-4] have been shown to impact users' performance and satisfaction with a given visualization. These findings suggest that it can be valuable to develop visualization systems that can provide personalized support targeting specific user characteristics. Furthermore, recent research [e.g., 3,5] has shown that eye tracking data can be leveraged to identify the elements of a visualization for which specific user differences hinder user experience or performance, thus providing concrete information on which specific personalized support could be helpful for different users (e.g., users with low perceptual speed may benefit from help in processing legends [1]). Though these findings are encouraging toward the design of user-adaptive or customized visualizations, they are generally related to either fictional tasks or research prototypes. So, it is unclear if existing results on the value of user-adaptive visualizations can transfer to real-world settings.

This paper summarizes an ongoing multi-year project aiming to uncover knowledge and techniques for devising intelligent environments for user-adaptive visualizations. We ran three studies designed to investigate the impact of user and task characteristics on user performance and satisfaction in different visualization contexts. Eye-tracking data collected in each study was analyzed to uncover possible interactions between user/task characteristics and gaze behavior during visualization processing. Finally, we investigated user models that can assess user characteristics relevant for adaptation from eye tracking data.

A user-adaptive information visualization system capable of learning models of users and the visualization tasks they perform could provide interventions optimized for helping specific users in specific task contexts. In this paper, we investigate the accuracy of predicting visualization tasks, user performance on tasks, and user traits from gaze data. We show that predictions made with a logistic regression model are significantly better than a baseline classifier, with particularly strong results for predicting task type and user performance. Furthermore, we compare classifiers built with interface-independent and interface-dependent features, and show that the interface-independent features are comparable or superior to interface-dependent ones. Finally, we discuss how the accuracy of predictive models is affected if they are trained with data from trials that had highlighting interventions added to the visualization.

Recent rise of Massive Open Online Courses (MOOCs) with unlimited participants, makes employing learning tools such as interactive simulations all but inevitable. Interactive simulations give students the opportunity to experiment with concrete examples and develop better understanding of concepts they have learned. However, some students do not learn well from this relatively unstructured form of interaction, suggesting the provision of adaptive support as a way to address this issue. This paper presents a formal evaluation of providing support to facilitate open exploration. We describe the process of designing an intervention delivery mechanism for adding adaptive support to an exploratory interactive simulation. The experimental evaluation of the adaptive version of the simulation indicates that the adaptive support provided to students significantly improved their learning performance. Quantitative and qualitative evaluations of users' acceptance of the system are generally positive but pinpoint areas for improvement.

User performance and satisfaction when working with an interface is influenced by how quickly the user can acquire the skills necessary to work with the interface through practice. Learning curves are mathematical models that can represent a user's skill acquisition ability through parameters that describe the user's initial expertise as well as her learning rate. This information could be used by an interface to provide adaptive support to users who may otherwise be slow in learning the necessary skills. In this paper, we investigate the feasibility of predicting in real time a user's learning curve when working with ValueChart, an interactive visualization for decision making. Our models leverage various data sources (a user's gaze behavior, pupil dilation, cognitive abilities), and we show that they outperform a baseline that leverages only knowledge on user task performance so far. We also show that the best performing model achieves good accuracies in predicting users' learning curves even after observing users' performance only on a few tasks. These results are promising toward the design of user-adaptive visualizations that can dynamically support a user in acquiring the necessary skills to complete visual tasks.

There is increasing evidence that the effectiveness of information visualization techniques can be impacted by the particular needs and abilities of each user. This suggests that it is important to investigate information visualization systems that can dynamically adapt to each user. In this paper, we address the question of how to adapt. In particular, we present a study to evaluate a variety of visual prompts, called "interventions", that can be performed on a visualization to help users process it. Our results show that some of the tested interventions perform better than a condition in which no intervention is provided, both in terms of task performance as well as subjective user ratings. We also discuss findings on how intervention effectiveness is influenced by individual differences and task complexity.

A key challenge for information visualization designers lies in developing systems that best support users in terms of their individual abilities, needs, and preferences. However, most visualizations require users to first gather a certain set of skills before they can efficiently process the displayed information. This paper presents a first step towards designing visualizations that provide personalized support in order to ease the so-called 'learning curve' during a user's skill acquisition phase. We present prediction models, trained on users' gaze data, that can identify if users are still in the skill acquisition phase or if they have gained the necessary abilities. The paper first reveals that users exhibit the learning curve even during the usage of simple information visualizations, and then shows that we can generate reasonably accurate predictions about a user's skill acquisition using solely their eye gaze behavior.

Abstract There is increasing evidence that user characteristics can have a significant impact on visualization effectiveness, suggesting that visualizations could be designed to better fit each user's specific needs. Most studies to date, however, have looked at static visualizations. Studies considering interactive visualizations have only looked at a limited number of user characteristics, and consider either low‐level tasks (e.g., value retrieval), or high‐level tasks (in particular: discovery), but not both. This paper contributes to this line of work by looking at the impact of a large set of user characteristics on user performance with interactive visualizations, for both low and high‐level tasks. We focus on interactive visualizations that support decision making, exemplified by a visualization known as Value Charts. We include in the study two versions of ValueCharts that differ in terms of layout, to ascertain whether layout mediates the impact of individual differences and could be considered as a form of personalization. Our key findings are that (i) performance with low and high‐level tasks is affected by different user characteristics, and (ii) users with low visual working memory perform better with a horizontal layout. We discuss how these findings can inform the provision of personalized support to visualization processing.

Information visualization systems have traditionally followed a one-size-fits-all model, typically ignoring an individual user's needs, abilities, and preferences. However, recent research has indicated that visualization performance could be improved by adapting aspects of the visualization to the individual user. To this end, this article presents research aimed at supporting the design of novel user-adaptive visualization systems. In particular, we discuss results on using information on user eye gaze patterns while interacting with a given visualization to predict properties of the user's visualization task; the user's performance (in terms of predicted task completion time); and the user's individual cognitive abilities, such as perceptual speed, visual working memory, and verbal working memory. We provide a detailed analysis of different eye gaze feature sets, as well as over-time accuracies. We show that these predictions are significantly better than a baseline classifier even during the early stages of visualization usage. These findings are then discussed with a view to designing visualization systems that can adapt to the individual user in real time.

We present the results of a study that explored the emotions experienced by students during interaction with an educational game for math (Heroes of Math Island). Starting from emotion frameworks in affective computing and education, we considered a larger set of emotions than in related research. For emotion labeling, we employed a standard method that relies on trained judges to report emotions over 20-second intervals. However, we asked judges to report all observed emotions in each interval, as opposed to only choosing one, as is standard practice. This variation allows us to discuss the appropriateness of this interval for emotion labeling. We present a detailed analysis of inter-coder reliability, both aggregated and over individual students, that considers not only the matching by judges over emotion type, but also the number of emotions detected.

There is increasing evidence that users' characteristics such as cognitive abilities and personality have an impact on the effectiveness of information visualization techniques. This paper investigates the relationship between such characteristics and fine-grained user attention patterns. In particular, we present results from an eye tracking user study involving bar graphs and radar graphs, showing that a user's cognitive abilities such as perceptual speed and verbal working memory have a significant impact on gaze behavior, both in general and in relation to task difficulty and visualization type. These results are discussed in view of our long-term goal of designing information visualisation systems that can dynamically adapt to individual user characteristics.

Information Visualization systems have traditionally followed a one-size-fits-all model, typically ignoring an individual user's needs, abilities and preferences. However, recent research has indicated that visualization performance could be improved by adapting aspects of the visualization to each individual user. To this end, this paper presents research aimed at supporting the design of novel user-adaptive visualization systems. In particular, we discuss results on using information on user eye gaze patterns while interacting with a given visualization to predict the user's visualization tasks, as well as user cognitive abilities including perceptual speed, visual working memory, and verbal working memory. We show that such predictions are significantly better than a baseline classifier even during the early stages of visualization usage. These findings are discussed in view of designing visualization systems that can adapt to each individual user in real-time.

Visual and auditory cues are important facilitators of user engagement in virtual environments and video games. Prior research supports the notion that our perception of visual fidelity (quality) is influenced by auditory stimuli. Understanding exactly how our perception of visual fidelity changes in the presence of multimodal stimuli can potentially impact the design of virtual environments, thus creating more engaging virtual worlds and scenarios. Stereoscopic 3-D display technology provides the users with additional visual information (depth into and out of the screen plane). There have been relatively few studies that have investigated the impact that auditory stimuli have on our perception of visual fidelity in the presence of stereoscopic 3-D. Building on previous work, we examine the effect of auditory stimuli on our perception of visual fidelity within a stereoscopic 3-D environment.

No abstract available.

This workshop addresses a wide range of issues concerning eye gaze: recognizing user's gaze, generating gaze behaviors in conversational humanoids, analyzing human attentional behaviors during interacting with IUIs, and evaluation of gaze-based IUIs. Through a comprehensive discussion, the workshop aims at bringing together researchers with different backgrounds, and establishing an interdisciplinary research community in attention aware interactive systems.

In this paper, we discuss efforts to improve the adaptive feedback provided by a pedagogical agent for an educational-game on number factorization. The improvements targeted both the form of the feedback as well as the accuracy of the student model that the agent uses to decide when and how to intervene. We present preliminary results of a study to evaluate the impact of this improved feedback on the game's pedagogical effectiveness

Workshop jointly chaired by Cristina Conati, and Tania Mitrovic.

In this paper, we describe a user study aimed at evaluating the effectiveness of two different data visualization techniques developed for describing complex environmental changes in an interactive system designed to foster awareness in sustainable development. While several studies have compared alternative visualizations, the distinguishing feature of our research is that we try to understand whether individual user differences may be used as predictors of visualization effectiveness in choosing among alternative visualizations for a given task. We show that the cognitive ability known as perceptual speed can predict which one of our target visualizations is most effective for a given user. This result suggests that tailored visualization selection can be an effective way to improve user performance.

Interactive algorithm visualizations (AVs) are powerful tools for teaching and learning concepts that are difficult to describe with static media alone. However, while countless AVs exist, their widespread adoption by the academic community has not occurred due to usability problems and mixed results of pedagogical effectiveness reported in the AV and education literature. This paper presents our experiences designing and evaluating CIspace, a set of interactive AVs for demonstrating fundamental Artificial Intelligence algorithms. In particular, we first review related work on AVs and theories of learning. Then, from this literature, we extract and compile a taxonomy of goals for designing interactive AVs that address key pedagogical and usability limitations of existing AVs. We advocate that differentiating between goals and design features that implement these goals will help designers of AVs make more informed choices, especially considering the abundance of often conflicting and inconsistent design recommendations in the AV literature. We also describe and present the results of a range of evaluations that we have conducted on CIspace that include semi-formal usability studies, usability surveys from actual students using CIspace as a course resource, and formal user studies designed to assess the pedagogical effectiveness of CIspace in terms of both knowledge gain and user preference. Our main results show that (i) studying with our interactive AVs is at least as effective at increasing student knowledge as studying with carefully designed paper-based materials; (ii) students like using our interactive AVs more than studying with the paper-based materials; (iii) students use both our interactive AVs and paper-based materials in practice although they are divided when forced to choose between them; (iv) students find our interactive AVs generally easy to use and useful. From these results, we conclude that while interactive AVs may not be universally preferred by students, it is beneficial to offer a variety of learning media to students to accommodate individual learning preferences. We hope that our experiences will be informative for other developers of interactive AVs, and encourage educators to exploit these potentially powerful resources in classrooms and other learning environments.

In this research, we outline a user modeling framework that uses both unsupervised and supervised machine learning in order to reduce development costs of building user models, and facilitate transferability. We apply the framework to model student learning during interaction with the Adaptive Coach for Exploration (ACE) learning environment (using both interface and eye-tracking data). In addition to demonstrating framework effectiveness, we also compare results from previous research on applying the framework to a different learning environment and data type. Our results also confirm previous research on the value of using eye-tracking data to assess student learning.

We describe a mixed-initiative framework designed to sup-port the customization of complex graphical user interfaces. The framework uses an innovative form of online GOMS analysis to provide the user with tailored customization sug-gestions aimed at maximizing the user’s performance with the interface. The suggestions are presented non-intrusively, minimizing disruption and allowing the user to maintain full control. The framework has been applied to a general user-productivity application. A formal user evaluation of the system provides encouraging evidence that this mixed-initiative approach is preferred to a purely adaptable alterna-tive and that the system’s suggestions help improve task per-formance.

In this paper we describe research on using eye-tracking data for on-line assessment of user meta-cognitive behavior during the interaction with an intelligent learning environment. We describe the probabilistic user model that processes this information, and its formal evaluation. We show that adding eye-tracker information significantly improves the model accuracy on assessing user exploration and self-explanation behaviors.

This paper describes the design of the CIspace interactive visualization tools for teaching and learning Artificial Intelligence. Our approach to design is to iterate through three phases: identifying pedagogical and usability goals for supporting both educators and students, designing to achieve these goals, and then evaluating our system. We believe identifying these goals is essential in confronting the usability deficiencies and mixed results about the pedagogical effectiveness of interactive visualizations reported in the Education literature. The CIspace tools have been used and positively received in undergraduate and graduate classrooms at the University of British Columbia and internationally. We hope that our experiences can inform other developers of interactive visualizations and encourage their use in classrooms and other learning environments.

There has been an increasing interest in exploring how recognition of a user's affective state can be exploited in creating more effective human-computer interaction. It has been argued that IUIs may be able to improve interaction by including affective elements in their communication with the user (e.g. by showing empathy via adequate phrasing of feedback.) This workshop will address a variety of issues related to the development of what we will call the affective loop: detection/modeling of relevant user's states, selection of appropriate system responses (including responses that are designed to influence the user affective state but are not overtly affective), as well as synthesis of the appropriate affective expressions.

As computer applications become larger with every new version, there is a growing need to provide some way for users to manage the interface complexity. There are three different potential solutions to this problem: 1) an adaptable interface that allows users to customize the application to suit their needs; 2) an adaptive interface that performs the adaptation for the users; or 3) a combination of the adaptive and adaptable solutions, an approach that would be suitable in situations where users are not customizing effectively on their own. In this paper we examine what it means for users to engage in effective customization of a menu-based graphical user interface. We examine one aspect of effective customization, which is how characteristics of the users' tasks and customization behaviour affect their performance on those tasks. We do so by using a process model simulation based on cognitive modelling that generates quantitative predictions of user performance. Our results show that users can engage in customization behaviours that vary in efficiency. We use these results to suggest how adaptive support could be added to an adaptable interface to improve the effectiveness of the users' customization.

Electronic educational games can be highly entertaining, but studies have shown that they do not always trigger learning. To enhance the effectiveness of educational games, we propose intelligent pedagogical agents that can provide individualized instruction integrated with the entertaining nature of the games. In this paper, we describe one such agent, that we have developed for Prime Climb, an educational game on number factorization. The Prime Climb agent relies on a probabilistic student model to generate tailored interventions aimed at helping students learn number factorization through the game. After describing the functioning of the agent and the underlying student model, we report the results of an empirical study that we performed to test the agent's effectiveness.

We present an adaptive interface designed to provide tailored support for the understanding of written instructional material. The interface relies on a user model based on a Bayesian network, that assesses users' understanding as users read the instructional material and try to understand it by generating explanations to themselves. The user model's assessment is used by the interface to generate tailored scaffolding of further user's explanations that can improve the user's comprehension. After illustrating how the Bayesian user model assesses understanding from the user's explanations and from latency data on the user's attention, we discuss initial results on the effectiveness of the interface's adaptive interventions.