Parkinson's Disease (PD) significantly impacts driving abilities, often leading to early driving cessation or accidents due to reduced CHI '25, Yokohama, Japan

Older adults often struggle to meet their psychological needs due to retirement and living alone. Recent studies suggest that games featuring emotional challenge (EC) can help fulfill basic psychological needs such as autonomy, competence, and relatedness by facilitating emotional exploration. However, it remains unclear whether older adults can benefit from EC games, whether they find this genre enjoyable, and how these games should be designed to better meet their needs. This work explores older adults' experiences and perceptions of playing EC games through two studies. The first study involved playing Detroit: Become Human, revealing that older adults derived multifaceted psychological experiences from playing the game. The second study involved a custom-designed game scenario tailored to older adults, demonstrating that meaningful choices significantly influenced autonomy need satisfaction. Based on these findings, we offer five design guidelines for developing EC games that satisfy psychological needs of older adults.

Despite the growing exploration of body games for social interaction, current approaches predominantly focus on physical mimicry while overlooking the critical emotional dimension of bodily expression. We address this gap by investigating emotion mimicry as a novel game mechanic for enhancing social presence and interpersonal connection. This paper introduces EmoEcho, a two-player 2D side-scrolling game that integrates emotion mimicry into its core mechanics. In EmoEcho, one player's facial expressions trigger game events while the second player must mimic these expressions to interact with game elements. Through a controlled comparative study (N=24, 12 pairs) contrasting emotion-based input against conventional keyboard controls, we demonstrate that emotion mimicry significantly enhanced social presence and overall game experience. Qualitative interviews further suggest that emotion mimicry enriched player experience via intriguing emotion-based embodied interaction, collaborative emotion co-creation, and development of shared emotional experiences. Our findings extend the understanding of social body games beyond physical mimicry to the emotional domain and provide actionable design implications for creating emotional social games. This work opens new space for game designers to leverage face-to-face emotional expressions as a powerful interface for creating meaningful social play experiences.

Animating virtual characters has always been a fundamental research problem in virtual reality (VR). Facial animations play a crucial role as they effectively convey emotions and attitudes of virtual humans. However, creating such facial animations can be challenging, as current methods often involve utilization of expensive motion capture devices or significant investments of time and effort from human animators in tuning animation parameters. In this paper, we propose a holistic solution to automatically animate virtual human faces. In our solution, a deep learning model was first trained to retarget the facial expression from input face images to virtual human faces by estimating the blendshape coefficients. This method offers the flexibility of generating animations with characters of different appearances and blendshape topologies. Second, a practical toolkit was developed using Unity 3D, making it compatible with the most popular VR applications. The toolkit accepts both image and video as input to animate the target virtual human faces and enables users to manipulate the animation results. Furthermore, inspired by the spirit of Human-in-the-loop (HITL), we leveraged user feedback to further improve the performance of the model and toolkit, thereby increasing the customization properties to suit user preferences. The whole solution, for which we will make the code public, has the potential to accelerate the generation of facial animations for use in VR applications. https://github.com/showlab/BYOC

As video lectures are gaining more popularity, determining their effectiveness and obtaining valuable feedback have become necessary. To measure the learners’ attention state during video lectures, we specified the conceptual “with-me-ness” (WMN) as slide-level WMN (SL-WMN). The content domain on each slide was automatically extracted via an optical character recognition (OCR)-based method, while the eye gazing behaviors were analyzed through a Gaussian mixture modeling (GMM) fixation clustering method. Both domain-specific WMN and behavior-enriched WMN were then computed via OCR- and GMM-OCR-based methods to measure the learners’ attention levels. We conducted an experiment to collect in-lecture eye-tracking data, video recordings, and post-lecture test scores from 50 Grade 8 students. The results demonstrated that both OCR- and GMM-OCR-based SL-WMNs are reliable and compatible automatic measurements of learners’ attention states during video lectures. A survey from participating learners and lecturers also revealed highly favorable feedback for the developed SL-WMNs.

DailyConnect is a visual-based mobile application that supports children with autism spectrum disorder (ASD) in recalling memories by reviewing photos through discrete trial training (DTT) to understand situation-based emotions. To assess DailyConnect and its adaptability to a child’s characteristics and emotional situations, a pilot study was conducted that included 15 children with ASD and their parents and teachers. The DTT steps—memory recall and situation recognition, emotion recognition, emotion cues, facial expression recognition, and response behavior—were reliable in assessing the understanding of emotional situations when compared before and after the intervention in four categories of emotional situations, namely, happiness, sadness, anger, and fear. The results revealed that DailyConnect improves the understanding of situation-based emotions, particularly negative emotions (e.g., sadness: mean diff = −.687, sig. < .01, T = −3.866, d = 1.006; anger: mean diff = −.952, sig. < .01, T = −6.187, d = .705; fear: mean diff = −.961, sig. < .01, T = −5.522, d = .627); however, its effectiveness varied for children in different emotional situations. Furthermore, subjective feedback from participants and users (parents and teachers) provided insights into design considerations for similar mobile aids.

Target selection is a crucial task in augmented reality (AR). Recent evidence suggests that user motion can significantly influence target selection. However, no systematic research has been conducted on target selection within varied intensity user motions and AR settings. This paper was carried out to investigate the effects of four user motions (i.e., standing, walking, running, and jumping) and two viewing modes (i.e., viewpoint-dependent and viewpoint-independent) on user performance of target selection in AR. Two typical selection techniques (i.e., virtual hand and ray-casting) were utilized for short-range and long-range selection tasks, respectively. Our results indicate that the target selection performance decreased as the intensity of user motion increased, and users demonstrated better performance in the viewpoint-independent mode than in the viewpoint-dependent mode. We also observed that users took a longer amount of time to select targets when using the ray-casting technique than the virtual hand technique. We conclude with a set of design guidelines to improve the AR target selection performance of users while in motion.

The blossoming semi-automated vehicles allow drivers to engage in various non-driving-related tasks, which may stimulate diverse emotions, thus affecting takeover safety. Though the effects of emotion on takeover behavior have recently been examined, how to effectively obtain and utilize drivers' emotions for predicting takeover behavior remains largely unexplored. We propose EmoTake, a deep learning-empowered system that explores drivers' emotional and physical states to predict takeover readiness, reaction time, and quality. The key enabler is a deep neural framework that extracts drivers' fine-grained body movements from a camera and interprets them into drivers' multi-channel emotional and physical information (e.g., facial expression, and head pose) for prediction. Our study (N = 26) verifies the efficiency of EmoTake and shows that: 1) facial expression benefits prediction; 2) emotions have diverse impacts on takeovers. Our findings provide insights into takeover prediction and in-vehicle emotion regulation.

The global trend of population aging poses significant challenges to society and healthcare systems, particularly because of neurocognitive disorders (NCDs) such as Parkinson's disease (PD) and Alzheimer's disease (AD). In this context, artificial intelligence techniques have demonstrated promising potential for the objective assessment and detection of NCDs. Multimodal contactless screening technologies, such as speech-language processing, computer vision, and virtual reality, offer efficient and convenient methods for disease diagnosis and progression tracking. This paper systematically reviews the specific methods and applications of these technologies in the detection of NCDs using data collection paradigms, feature extraction, and modeling approaches. Additionally, the potential applications and future prospects of these technologies for the detection of cognitive and motor disorders are explored. By providing a comprehensive summary and refinement of the extant theories, methodologies, and applications, this study aims to facilitate an in-depth understanding of these technologies for researchers, both within and outside the field. To the best of our knowledge, this is the first survey to cover the use of speech-language processing, computer vision, and virtual reality technologies for the detection of NSDs.

Challenge is the core element of digital games. The wide spectrum of physical, cognitive, and emotional challenge experiences provided by modern digital games can be evaluated subjectively using a questionnaire, the CORGIS, which allows for a post hoc evaluation of the overall experience that occurred during game play. Measuring this experience dynamically and objectively, however, would allow for a more holistic view of the moment-to-moment experiences of players. This study, therefore, explored the potential of detecting perceived challenge from physiological signals. For this, we collected physiological responses from 32 players who engaged in three typical game scenarios. Using perceived challenge ratings from players and extracted physiological features, we applied multiple machine learning methods and metrics to detect challenge experiences. Results show that most methods achieved a detection accuracy of around 80%. We discuss in-game challenge perception, challenge-related physiological indicators and AI-supported challenge detection to inform future work on challenge evaluation.

Modeling the endpoint uncertainty of moving target selection with crossing is essential to understand factors such as speed-accuracy trade-off and interaction efficiency in crossing-based user interfaces with dynamic contents. However, there have been few studies looking into this research topic in the HCI field. This paper presents a Quaternary-Gaussian model to quantitatively measure the endpoint uncertainty in crossing-based moving target selection. To validate this model, we conducted an experiment with discrete crossing tasks on five factors, i.e., initial distance, size, speed, orientation, and moving direction. Results showed that our model fit the data of μ and σ accurately with adjusted R2 of 0.883 and 0.920. We also demonstrated the validity of our model in predicting error rates in crossing-based moving target selection. We concluded with a set of implications for future designs.

Recent work introduced the notion of 'emotional challenge' promising for understanding more unique and diverse player experiences (PX). Although emotional challenge has immediately attracted HCI researchers' attention, the concept has not been experimentally explored, especially in virtual reality (VR), one of the latest gaming environments. We conducted two experiments to investigate how emotional challenge affects PX when separately from or jointly with conventional challenge in VR and PC conditions. We found that relatively exclusive emotional challenge induced a wider range of different emotions in both conditions, while the adding of emotional challenge broadened emotional responses only in VR. In both experiments, VR significantly enhanced the measured PX of emotional responses, appreciation, immersion and presence. Our findings indicate that VR may be an ideal medium to present emotional challenge and also extend the understanding of emotional (and conventional) challenge in video games.

Most intervention methods used for social skills training in children with autism are dedicated to high-functioning autism (HFA). However, extensive neurological and developmental disorders of low-functioning autism (LFA) have hampered their adoption. In this study, we observed and interviewed children with LFA, and their teachers, from a local educational institution, to better understand the children's social needs and barriers. Then, with the aim of aiding the children with their social activities, we illustrate the design process of SqueeBall, an interactive playground equipment. We evaluated the design with 18 children (16 with LFA and 2 with HFA) between 2.5 and 7 years of age. Results showed that these children had a pleasant game experience when the group bonded, and the equipment had a positive effect on aiding them in various ways. Finally, we discuss the challenges and opportunities of multimedia interaction techniques in aiding children with LFA.

Second language (L2) pronunciation training has been a worldwide task. Although computer technology makes it possible to develop a talking head to teach pronunciation like a real language teacher, little is known about how a talking head may act on L2 learners’ emotional and cognitive learning process. We investigate L2 learners’ achievement emotions, cognitive load, and pronunciation learning performance in a computer-assisted pronunciation training (CAPT) system embedded with four conditions: audio only (AU), a human face (HF), a 3D talking head with front view (3Df), and a 3D talking head with both front and profile views (3D). Results showed that, with learning time went on, participants’ perceived anxiety, boredom, and pride increased while shame and hopelessness decreased and enjoyment kept stable. With 3D, participants’ anxiety increased the most and boredom increased the least. Moreover, 3D group also perceived the highest germane load and got the highest pronunciation learning performance. Furthermore, anxiety and shame correlated with learning performance positively while boredom correlated with it negatively; enjoyment and pride correlated positively with performance on Mandarin tones. These findings significantly contribute to the efforts to design or select virtual characters for computer-aided language learning (CALL) and also provide a valuable reference to study achievement emotions in HCI systems.