Recent breakthroughs in Generative AI (GenAI) are reshaping educational landscapes, presenting challenges and opportunities. While all contexts present unique challenges, rural schools are historically under-resourced, facing persistent technology-related barriers. To understand and reduce these barriers, we studied 31 rural high school educators across three U.S. states to examine their use of GenAI and understand how GenAI introduces new challenges, opportunities, and may exacerbate existing educational barriers. Results show while rural educators use GenAI to streamline teaching tasks, existing resource disparities restrict meaningful integration. Through rural educators’ voices, we reveal issues like infrastructure barriers, resistance to adoption, and lack of AI literacy training create significant obstacles. Nonetheless, educators envision GenAI can support themselves and their students, but findings emphasize the need for rural-specific design approaches. As a community, embracing inclusive GenAI design and re-examining assumptions about technology adoption in under-served educational contexts is essential to reducing barriers rather than widening them. Supplemental Material is open-sourced and available at https://osf.io/8hckv/.

Artificial Intelligence (AI) holds significant potential for supporting clinical decision-making, particularly in high-pressure environments, such as Cardiothoracic Intensive Care Units (CT-ICU). Care teams in these settings face challenges such as alarm fatigue, rapid staff turnover, time-sensitive decisions, and an overwhelming amount of data. AI-driven Clinical Decision-Support Systems (AI-CDSS) can support care teams in overcoming some of these challenges by providing solutions like detecting and reporting risk scores for adverse events that may lead to increased fatalities or re-admissions, enabling timely intervention. One key challenge with risk scores is missing data, which can create considerable uncertainty in risk score values. AI-CDSSs rarely convey the risk score uncertainty, which is important in the effectiveness and reliability of clinical decision-making. In this paper, we describe the interface design process for HEART, an AI-powered system developed collaboratively with clinical and AI experts over a 16-month iterative design process for a hospital’s CT-ICU. The HEART interactive interface integrates understandable visualizations of risk scores and their uncertainty within both a holistic view of all patients in the unit and detailed patient-specific views. We reflect on the user-centered design process, report findings from an expert walkthrough study, and discuss lessons learned as well as broader implications. This work contributes valuable insights into uncertainty visualization design for AI-derived risk scores in a critical care application. Beyond these specific insights, our work illustrates the kind of comprehensive, human-centered design process necessary for responsible AI adoption in critical environments.

Head tracking is commonly used in VR applications to allow users to naturally view 3D content using physical head movement, but many applications also support turning with hand-held controllers. Controller and joystick controls are convenient for practical settings where full 360-degree physical rotation is not possible, such as when the user is sitting at a desk. Though controller-based rotation provides the benefit of convenience, previous research has demonstrated that virtual or joystick-controlled view rotation to have drawbacks of sickness and disorientation compared to physical turning. To combat such issues, researchers have considered various techniques such as speed adjustments or reduced field of view, but data is limited on how different variations for joystick rotation influences sickness and orientation perception. Our studies include different variations of techniques such as joystick rotation, resetting, and field-of-view reduction. We investigate trade-offs among different techniques in terms of sickness and the ability to maintain spatial orientation. In two controlled experiments, participants traveled through a sequence of rooms and were tested on spatial orientation, and we also collected subjective measures of sickness and preference. Our findings indicate a preference by users towards directly-manipulated joystick-based rotations compared to user-initiated resetting and minimal effects of technique on spatial awareness.

We consider the following video activity recognition (VAR) task: given a video, infer the set of activities being performed in the video and assign each frame to an activity. Although VAR can be solved accurately using existing deep learning techniques, deep networks are neither interpretable nor explainable and as a result their use is problematic in high stakes decision-making applications (in healthcare, experimental Biology, aviation, law, etc.). In such applications, failure may lead to disastrous consequences and therefore it is necessary that the user is able to either understand the inner workings of the model or probe it to understand its reasoning patterns for a given decision. We address these limitations of deep networks by proposing a new approach that feeds the output of a deep model into a tractable, interpretable probabilistic model called a dynamic conditional cutset network that is defined over the explanatory and output variables and then performing joint inference over the combined model. The two key benefits of using cutset networks are: (a) they explicitly model the relationship between the output and explanatory variables and as a result, the combined model is likely to be more accurate than the vanilla deep model and (b) they can answer reasoning queries in polynomial time and as a result, they can derive meaningful explanations by efficiently answering explanation queries. We demonstrate the efficacy of our approach on two datasets, Textually Annotated Cooking Scenes (TACoS), and wet lab, using conventional evaluation measures such as the Jaccard Index and Hamming Loss, as well as a human-subjects study.

While EXplainable Artificial Intelligence (XAI) approaches aim to improve human-AI collaborative decision-making by improving model transparency and mental model formations, experiential factors associated with human users can cause challenges in ways system designers do not anticipate. In this article, we first showcase a user study on how anchoring bias can potentially affect mental model formations when users initially interact with an intelligent system and the role of explanations in addressing this bias. Using a video activity recognition tool in cooking domain, we asked participants to verify whether a set of kitchen policies are being followed, with each policy focusing on a weakness or a strength. We controlled the order of the policies and the presence of explanations to test our hypotheses. Our main finding shows that those who observed system strengths early on were more prone to automation bias and made significantly more errors due to positive first impressions of the system, while they built a more accurate mental model of the system competencies. However, those who encountered weaknesses earlier made significantly fewer errors, since they tended to rely more on themselves, while they also underestimated model competencies due to having a more negative first impression of the model. Motivated by these findings and similar existing work, we formalize and present a conceptual model of user’s past experiences that examine the relations between user’s backgrounds, experiences, and human factors in XAI systems based on usage time. Our work presents strong findings and implications, aiming to raise the awareness of AI designers toward biases associated with user impressions and backgrounds.

EXplainable Artificial Intelligence (XAI) approaches are used to bring transparency to machine learning and artificial intelligence models, and hence, improve the decision-making process for their end-users. While these methods aim to improve human understanding and their mental models, cognitive biases can still influence a user’s mental model and decision-making in ways that system designers do not anticipate. This paper presents research on cognitive biases due to ordering effects in intelligent systems. We conducted a controlled user study to understand how the order of observing system weaknesses and strengths can affect the user’s mental model, task performance, and reliance on the intelligent system, and we investigate the role of explanations in addressing this bias. Using an explainable video activity recognition tool in the cooking domain, we asked participants to verify whether a set of kitchen policies are being followed, with each policy focusing on a weakness or a strength. We controlled the order of the policies and the presence of explanations to test our hypotheses. Our main finding shows that those who observed system strengths early-on were more prone to automation bias and made significantly more errors due to positive first impressions of the system, while they built a more accurate mental model of the system competencies. On the other hand, those who encountered weaknesses earlier made significantly fewer errors since they tended to rely more on themselves, while they also underestimated model competencies due to having a more negative first impression of the model. Our work presents strong findings that aim to make intelligent system designers aware of such biases when designing such tools.

The visualization of hierarchically structured data over time is an ongoing challenge and several approaches exist trying to solve it. Techniques such as animated or juxtaposed tree visualizations are not capable of providing a good overview of the time series and lack expressiveness in conveying changes over time. Nested streamgraphs provide a better understanding of the data evolution, but lack the clear outline of hierarchical structures at a given timestep. Furthermore, these approaches are often limited to static hierarchies or exclude complex hierarchical changes in the data, limiting their use cases. We propose a novel visual metaphor capable of providing a static overview of all hierarchical changes over time, as well as clearly outlining the hierarchical structure at each individual time step. Our method allows for smooth transitions between treemaps and nested streamgraphs, enabling the exploration of the trade-off between dynamic behavior and hierarchical structure. As our technique handles topological changes of all types, it is suitable for a wide range of applications. We demonstrate the utility of our method on several use cases, evaluate it with a user study, and provide its full source code.

Mixed-initiative systems allow users to interactively provide feedback to potentially improve system performance. Human feedback can correct model errors and update model parameters to dynamically adapt to changing data. Additionally, many users desire the ability to have a greater level of control and fix perceived flaws in systems they rely on. However, how the ability to provide feedback to autonomous systems influences user trust is a largely unexplored area of research. Our research investigates how the act of providing feedback can affect user understanding of an intelligent system and its accuracy. We present a controlled experiment using a simulated object detection system with image data to study the effects of interactive feedback collection on user impressions. The results show that providing human-in-the-loop feedback lowered both participants’ trust in the system and their perception of system accuracy, regardless of whether the system accuracy improved in response to their feedback. These results highlight the importance of considering the effects of allowing end-user feedback on user trust when designing intelligent systems.

It is becoming increasingly common for intelligent systems, such as recommender systems, to provide explanations for their generated recommendations to the users. However, we still do not have a good understanding of what types of explanations work and what factors affect the effectiveness of different types of explanations. Our work focuses on explanations for movie recommender systems. This paper presents a mixed study where we hypothesize that the type of explanation, as well as user motivation for watching movies, will affect how users respond to recommendation system explanations. Our study compares three types of explanations: i) neighbor-ratings, ii) profile-based, and iii) event-based, as well as three types of user movie-watching motivations: i) hedonic (fun and relaxation), ii) eudaimonic (inspiration and meaningfulness), and iii) educational (learning new content). We discuss the implications of the study results for the design of explanations for movie recommender systems, and future novel research directions that the study results uncover.

Domain-specific intelligent systems are meant to help system users in their decision-making process. Many systems aim to simultaneously support different users with varying levels of domain expertise, but prior domain knowledge can affect user trust and confidence in detecting system errors. While it is also known that user trust can be influenced by first impressions with intelligent systems, our research explores the relationship between ordering bias and domain expertise when encountering errors in intelligent systems. In this paper, we present a controlled user study to explore the role of domain knowledge in establishing trust and susceptibility to the influence of first impressions on user trust. Participants reviewed an explainable image classifier with a constant accuracy and two different orders of observing system errors (observing errors in the beginning of usage vs. in the end). Our findings indicate that encountering errors early-on can cause negative first impressions for domain experts, negatively impacting their trust over the course of interactions. However, encountering correct outputs early helps more knowledgable users to dynamically adjust their trust based on their observations of system performance. In contrast, novice users suffer from over-reliance due to their lack of proper knowledge to detect errors.

Machine learning and artificial intelligence algorithms can assist human decision making and analysis tasks. While such technology shows promise, willingness to use and rely on intelligent systems may depend on whether people can trust and understand them. To address this issue, researchers have explored the use of explainable interfaces that attempt to help explain why or how a system produced the output for a given input. However, the effects of meaningful and meaningless explanations (determined by their alignment with human logic) are not properly understood, especially with users who are non-experts in data science. Additionally, we wanted to explore how explanation inclusion and level of meaningfulness would affect the user’s perception of accuracy. We designed a controlled experiment using an image classification scenario with local explanations to evaluate and better understand these issues. Our results show that whether explanations are human-meaningful can significantly affect perception of a system’s accuracy independent of the actual accuracy observed from system usage. Participants significantly underestimated the system’s accuracy when it provided weak, less human-meaningful explanations. Therefore, for intelligent systems with explainable interfaces, this research demonstrates that users are less likely to accurately judge the accuracy of algorithms that do not operate based on human-understandable rationale.