This paper presents an N-ary Gaussian Model for predicting endpoint distributions in pointing tasks across task scenarios. Built on the foundational principles of the Ternary Gaussian model series, our model framework allows researchers to define parameter constraints and automatically refine model combinations, eliminating the need for predefined equations based on data analysis. We utilize the Bayesian Information Criterion (BIC) for model selection, ensuring simplicity while maintaining predictive accuracy. We conducted a comparative analysis against published baselines across 7 diverse datasets, covering 1D, 2D, and 3D tasks, different input modalities, different display devices, and time-constrained scenarios, demonstrating the robustness and generalization of the N-ary Gaussian Model. The N-ary Gaussion model offers an automated solution for modeling pointing uncertainty, and also incorporates cross output device, input modality, and temporal constraint factors into spatial pointing uncertainty modeling for the first time.

Virtual reality (VR) allows users to observe and manipulate 3D geometry from multiple viewpoints. Most VR selection work, however, optimizes techniques for selecting entire objects. Selecting a single face on a polyhedron remains underexplored and is more challenging because the interaction must act on a local component while preserving the object's global structure. We introduce a design space tailored to this task with three dimensions: viewing strategy, disambiguation consistency, and interaction metaphor. Guided by this space, we design eight freehand techniques for polyhedral face selection. A within-subjects study with 16 participants evaluates these techniques across polyhedral complexity (two radii; face counts 4, 6, and 12). The results identify three complementary top techniques, reveal tradeoffs between viewing choices and geometric preservation, and yield concrete guidelines for matching techniques to target geometry and task demands. A follow-up study with complex, realistic models confirms the robustness and practical usability of the three techniques. Together, these contributions shift attention from whole object selection to precise component selection in VR and provide actionable methods for 3D modeling, assembly, and texturing.

Parkinson's Disease (PD) significantly impacts driving abilities, often leading to early driving cessation or accidents due to reduced CHI '25, Yokohama, Japan

Older adults often struggle to meet their psychological needs due to retirement and living alone. Recent studies suggest that games featuring emotional challenge (EC) can help fulfill basic psychological needs such as autonomy, competence, and relatedness by facilitating emotional exploration. However, it remains unclear whether older adults can benefit from EC games, whether they find this genre enjoyable, and how these games should be designed to better meet their needs. This work explores older adults' experiences and perceptions of playing EC games through two studies. The first study involved playing Detroit: Become Human, revealing that older adults derived multifaceted psychological experiences from playing the game. The second study involved a custom-designed game scenario tailored to older adults, demonstrating that meaningful choices significantly influenced autonomy need satisfaction. Based on these findings, we offer five design guidelines for developing EC games that satisfy psychological needs of older adults.

For individuals in situations where their hands are occupied, no-handed touch interaction represents a significant yet underexplored area. In this paper, we investigate touch performance on screens using body parts (the nose, tongue, chin, elbow, and toe) rather than the hand. Experiments were conducted with 37 participants to analyze their touch interaction, where they selected targets using various body parts under different conditions. We employed the Dual-Gaussian model and Ternary-Gaussian model to describe the touch endpoints of these body parts. Based on the statistical criterion derived from the models, we proposed a no-handed assistive technology, which achieved up to a 24.4% improvement in tapping speed and a 58.47% increase in tapping accuracy. Additionally, this paper provides HCI developers with design guidelines for optimizing no-handed touch interaction interfaces.

Moving target selection in multimedia interactive systems faces unprecedented challenges as users increasingly interact across diverse, dynamic contexts-from live streaming in moving vehicles to VR gaming in varying environments. Existing approaches rely on probabilistic models that relate endpoint distribution to target properties (size, speed). However, these methods require substantial training data for each new context and lack transferability across scenarios, limiting their practical deployment in diverse multimedia environments where rich multimodal contextual information is readily available. This paper introduces MAGNeT (Multimodal Adaptive Gaussian Networks), which addresses these problems by combining classical statistical modeling with context-aware multimodal method. MAGNeT dynamically fuses pre-fitted Ternary-Gaussian models from various scenarios based on real-time contextual cues, enabling effective adaptation with minimal training data while preserving model interpretability. We take experiments on self-constructed 2D and 3D moving target selection datasets under in-vehicle vibration conditions. Extensive experiments demonstrate that MAGNeT achieves lower error rates with few-shot samples, by applying context-aware fusion of Gaussian experts from multi-factor conditions.

Proximity-based feedback provides users with real-time guidance as they approach an interaction goal. This type of feedback is particularly useful for tasks that require guidance during the interaction process, such as selecting moving targets. This work explores proximity-based feedback types and modalities to improve the selection of moving targets in VR by leveraging three feedback types that combine visual, auditory, and haptic modalities. We evaluated the performance of these mechanisms through two user studies, analyzing both objective data (e.g., selection time, error rate) and subjective data (e.g., user experience, preferences) to explore the characteristics of feedback types across different modalities and to examine the roles of various modalities within multimodal combinations. Our findings suggest optimal selection mechanisms for developers and should be tailored to different goals: achieving user precision, enabling quick movement to a target, considering task duration, and enhancing entertainment value. We also discuss applications that correspond to these different perspectives.

Multi-user locomotion in VR has grown increasingly common, posing numerous challenges. A key factor contributing to these challenges is the gaps in experience between active and passive users during co-locomotion. Yet, there remains a limited understanding of how and to what extent these experiential gaps manifest in diverse multi-user co-locomotion scenarios. This paper systematically explores the gaps in physiological and psychological experience indicators between active and passive users across various locomotion situations. Such situations include when active users walk, fly by joystick, or teleport, and passive users stand still or look around. We also assess the impact of factors such as sub-locomotion type, speed/teleport-interval, motion sickness susceptibility, etc. Accordingly, we delineate acceptability disparities between active and passive users, offering insights into leveraging notable experimental findings to mitigate discomfort during co-locomotion through avoidance or intervention.

Writing and editing mathematical expressions with complicated structures in computer system is difficult and time-consuming. To address this, we proposed MathAssist, a mathematical expression autocomplete technique that recommends full formulas in real-time based on the user’s input strokes. Our technique identifies user’s input purpose by matching the structure of the current user input to the structure of formulas in a database. To facilitate such process, we propose a novel tree-based formalization to represent formula. In comparison to a mathematical expression recognition algorithm (SRD) and a commercial MicroSoft Ink Equation (InkEqu), our approach outperformed both of them on task completion time (reduced by 37.14% and 37.58%) and accuracy (32.78% and 10.55% higher). We also discuss our findings in using autocomplete to assist formula editing.

Target selection is a crucial task in augmented reality (AR). Recent evidence suggests that user motion can significantly influence target selection. However, no systematic research has been conducted on target selection within varied intensity user motions and AR settings. This paper was carried out to investigate the effects of four user motions (i.e., standing, walking, running, and jumping) and two viewing modes (i.e., viewpoint-dependent and viewpoint-independent) on user performance of target selection in AR. Two typical selection techniques (i.e., virtual hand and ray-casting) were utilized for short-range and long-range selection tasks, respectively. Our results indicate that the target selection performance decreased as the intensity of user motion increased, and users demonstrated better performance in the viewpoint-independent mode than in the viewpoint-dependent mode. We also observed that users took a longer amount of time to select targets when using the ray-casting technique than the virtual hand technique. We conclude with a set of design guidelines to improve the AR target selection performance of users while in motion.

The global trend of population aging poses significant challenges to society and healthcare systems, particularly because of neurocognitive disorders (NCDs) such as Parkinson's disease (PD) and Alzheimer's disease (AD). In this context, artificial intelligence techniques have demonstrated promising potential for the objective assessment and detection of NCDs. Multimodal contactless screening technologies, such as speech-language processing, computer vision, and virtual reality, offer efficient and convenient methods for disease diagnosis and progression tracking. This paper systematically reviews the specific methods and applications of these technologies in the detection of NCDs using data collection paradigms, feature extraction, and modeling approaches. Additionally, the potential applications and future prospects of these technologies for the detection of cognitive and motor disorders are explored. By providing a comprehensive summary and refinement of the extant theories, methodologies, and applications, this study aims to facilitate an in-depth understanding of these technologies for researchers, both within and outside the field. To the best of our knowledge, this is the first survey to cover the use of speech-language processing, computer vision, and virtual reality technologies for the detection of NSDs.

Challenge is the core element of digital games. The wide spectrum of physical, cognitive, and emotional challenge experiences provided by modern digital games can be evaluated subjectively using a questionnaire, the CORGIS, which allows for a post hoc evaluation of the overall experience that occurred during game play. Measuring this experience dynamically and objectively, however, would allow for a more holistic view of the moment-to-moment experiences of players. This study, therefore, explored the potential of detecting perceived challenge from physiological signals. For this, we collected physiological responses from 32 players who engaged in three typical game scenarios. Using perceived challenge ratings from players and extracted physiological features, we applied multiple machine learning methods and metrics to detect challenge experiences. Results show that most methods achieved a detection accuracy of around 80%. We discuss in-game challenge perception, challenge-related physiological indicators and AI-supported challenge detection to inform future work on challenge evaluation.

This paper presents a Shape-Adaptive Ternary-Gaussian model for describing endpoint uncertainty when pointing at moving targets of arbitrary shapes. The basic idea of the model is to combine the uncertainty related to the target shape with the uncertainty caused by the target motion. First, we proposed a model to predict endpoint distribution on static targets based on a Dual-Space Decomposition (DUDE) algorithm. Then, we linearly combined a 2D Ternary-Gaussian model with the newly proposed DUDE-based model to make the 2D Ternary-Gaussian model adaptable to moving targets with random shapes. To verify the performance of our model, we compared it with the original 2D Ternary-Gaussian model and a recent proposed Inscribed Circle model in predicting endpoint distribution. The results show that the proposed model outperformed the two baseline models while maintaining good robustness across different shapes and moving speeds.

With more and more mobile devices (such as smart phones and tablets) supporting augmented reality (AR), using these devices to sketch in mid-air has become a popular application direction. However, due to the small display size of these devices, the user's field of view is limited and cannot see the context of graphics, which leads to the drawn graphics deviating from their expectations. In this article, we applied sonification technology to mid-air sketching with mobile AR device, and proposed a new method to address this problem. In our first experiment, we verified the feasibility of our method. Our experimental results showed that sonification can effectively reduce the deviation caused by a narrow field of view. In our second experiment, we further explored the application ability of this method in a wider range of sketching. Our experimental results showed that sonification can improve the aspect ratio of the drawn graphics. In addition, the results of the NASA-TLX questionnaire showed that the participants' mental demand and effort decreased significantly, and their subjective performance increased significantly. We proposed a new method, which can effectively improve the accuracy of mid-air sketching with a mobile AR device.

Task-irrelevant distractors affect visuo-motor control for target acquisition and studying such effects has already received much attention in human-computer interaction. However, there has been little research into distractor effects on crossing-based interaction. We thus conducted an empirical study on pen-based interfaces to investigate six crossing tasks with distractor interference in comparison to two tasks without it. The six distractor-related tasks differed in movement precision constraint (directional/amplitude), target size, target distance, distractor location and target-distractor spacing. We also developed and experimentally validated six quantitative models for the six tasks. Our results show that crossing targets with distractors had longer average times and similar accuracy than that without distractors. The effects of distractors varied depending on distractor location, target-distractor spacing and movement precision constraint. When spacing is smaller than 11.27 mm, crossing tasks with distractor interference can be regarded as pointing tasks or a combination of pointing and crossing tasks, which could be better fitted with our proposed models than Fitts’ law. According to these results, we provide practical implications to crossing-based user interface design.

Modeling error rates in moving target selection is critical in guiding the design and improving user performance in the user interface with dynamic contents. Despite the high accuracy of existing models in predicting error rates for various inputs, they need to fit a large number of samples for a specific interaction scenario. This paper presents a scenario adaptive model that can quickly learn the characterizes of endpoints from small data in a specific scenario, and then accurately predict the error rates in this scenario. We report on two studies evaluating the adaptability of the model in specific users (children and the older user) and pointing postures (walking and one-handed) with a smartphone. Significant improvements in error rates prediction of our model were observed compared to the previous non-adaptive one. We conclude with the findings from our results for insights into future interface design.

Modeling the endpoint uncertainty of moving target selection with crossing is essential to understand factors such as speed-accuracy trade-off and interaction efficiency in crossing-based user interfaces with dynamic contents. However, there have been few studies looking into this research topic in the HCI field. This paper presents a Quaternary-Gaussian model to quantitatively measure the endpoint uncertainty in crossing-based moving target selection. To validate this model, we conducted an experiment with discrete crossing tasks on five factors, i.e., initial distance, size, speed, orientation, and moving direction. Results showed that our model fit the data of μ and σ accurately with adjusted R2 of 0.883 and 0.920. We also demonstrated the validity of our model in predicting error rates in crossing-based moving target selection. We concluded with a set of implications for future designs.

The use of Virtual Reality (VR) in applications such as data analysis, artistic creation, and clinical settings requires high precision input. However, the current design of handheld controllers, where wrist rotation is the primary input approach, does not exploit the human fingers' capability for dexterous movements for high precision pointing and selection. To address this issue, we investigated the characteristics and potential of using a pen as a VR input device. We conducted two studies. The first examined which pen grip allowed the largest range of motion---we found a tripod grip at the rear end of the shaft met this criterion. The second study investigated target selection via 'poking' and ray-casting, where we found the pen grip outperformed the traditional wrist-based input in both cases. Finally, we demonstrate potential applications enabled by VR pen input and grip postures.

Recent work introduced the notion of 'emotional challenge' promising for understanding more unique and diverse player experiences (PX). Although emotional challenge has immediately attracted HCI researchers' attention, the concept has not been experimentally explored, especially in virtual reality (VR), one of the latest gaming environments. We conducted two experiments to investigate how emotional challenge affects PX when separately from or jointly with conventional challenge in VR and PC conditions. We found that relatively exclusive emotional challenge induced a wider range of different emotions in both conditions, while the adding of emotional challenge broadened emotional responses only in VR. In both experiments, VR significantly enhanced the measured PX of emotional responses, appreciation, immersion and presence. Our findings indicate that VR may be an ideal medium to present emotional challenge and also extend the understanding of emotional (and conventional) challenge in video games.

Flight itinerary ranking is critical for Online Travel Agencies (OTAs) since more and more customers book flights online. Currently, most OTAs still adopt rule-based strategies. However, rule-based methods are not able to model context-aware information and user preferences. To this end, a novel Personalized Flight itinerary Ranking Network (PFRN) is proposed in this paper. In PFRN, a Listwise Feature Encoding (LFE) structure is proposed to capture global context-aware information and mutual influences among inputs. Besides, we utilize behaviors of both individual user and group users sharing the same intention to express user preferences. Then a User Attention Mechanism is proposed to rank flight itineraries based on the user preferences effectively and efficiently. Offline experiments on real-world datasets from Amadeus and Fliggy show the superior performance of the proposed PFRN. Moreover, PFRN has been successfully deployed on online system for searching itineraries at Fliggy and achieved significant improvements.

Instant photo taking and sharing has become one of the most popular forms of social networking. However, taking high-quality photos is difficult as it requires knowledge and skill in photography that most non-expert users lack. In this paper we present SmartEye, a novel mobile system to help users take photos with good compositions in-situ. The back-end of SmartEye integrates the View Proposal Network (VPN), a deep learning based model that outputs composition suggestions in real time, and a novel, interactively updated module (P-Module) that adjusts the VPN outputs to account for personalized composition preferences. We also design a novel interface with functions at the front-end to enable real-time and informative interactions for photo taking. We conduct two user studies to investigate SmartEye qualitatively and quantitatively. Results show that SmartEye effectively models and predicts personalized composition preferences, provides instant high-quality compositions in-situ, and outperforms the non-personalized systems significantly.

Understanding the selection uncertainty of moving targets is a fundamental research problem in HCI. However, the only few works in this domain mainly focus on selecting 1D moving targets with certain input devices, where the model generalizability has not been extensively investigated. In this paper, we propose a 2D Ternary-Gaussian model to describe the selection uncertainty manifested in endpoint distribution for moving target selection. We explore and compare two candidate methods to generalize the problem space from 1D to 2D tasks, and evaluate their performances with three input modalities including mouse, stylus, and finger touch. By applying the proposed model in assisting target selection, we achieved up to 4% improvement in pointing speed and 41% in pointing accuracy compared with two state-of-the-art selection technologies. In addition, when we tested our model to predict pointing errors in a realistic user interface, we observed high fit of 0.94 R2.

With the development of virtual reality (VR) and human-computer interaction technology, how to use natural and efficient interaction methods in the virtual environment has become a hot topic of research. Gesture is one of the most important communication methods of human beings, which can effectively express users’ demands. In the past few decades, gesture-based interaction has made significant progress. This article focuses on the gesture interaction technology and discusses the definition and classification of gestures, input devices for gesture interaction, and gesture interaction recognition technology. The application of gesture interaction technology in virtual reality is studied, the existing problems in the current gesture interaction are summarized, and the future development is prospected.

Background Owing to recent advances in virtual reality (VR) technologies, effective user interaction with dynamic content in 3D scenes has become a research hotspot. Moving target selection is a basic interactive task in which the user performance research in tasks is significant to user interface design in VR. Different from the existing static target selection studies, the moving target selection in VR is affected by the change in target speed, angle and size, and lack of research on some key factors. Methods This study designs an experimental scenario in which the users play badminton under the condition of VR. By adding seven kinds of modal clues such as vision, audio, haptics, and their combinations, five kinds of moving speed and four kinds of serving angles, and the effect of these factors on the performance and subjective feelings in moving target selection in VR, is studied. Results The results show that the moving speed of the shuttlecock has a significant impact on the user performance. The angle of service has a significant impact on hitting rate, but has no significant impact on the hitting distance. The acquisition of the user performance by the moving target is mainly influenced by vision under the combined modalities; adding additional modalities can improve user performance. Although the hitting distance of the target is increased in the trimodal condition, the hitting rate decreases. Conclusion This study analyses the results of user performance and subjective perception, and then provides suggestions on the combination of modality clues in different scenarios.

Background Crossing-based target selection motion may attain less error rates and higher interactive speed in some cases. Most of the research in target selection fields are focused on the analysis of the interaction results. Additionally, as trajectories play a much more important role in crossing-based target selection compared to the other interactive techniques, an ideal model for trajectories can help computer designers make predictions about interaction results during the process of target selection rather than at the end of the whole process. Methods In this paper, a trajectory prediction model for crossing-based target selection tasks is proposed by taking the reference of a dynamic model theory. Results Simulation results demonstrate that our model performed well with regard to the prediction of trajectories, endpoints and hitting time for target-selection motion, and the average error of trajectories, endpoints and hitting time values were found to be 17.28%, 2.73mm and 11.50%, respectively.

In contrast to the extensive studies on static target pointing, much less formal understanding of moving target acquisition can be found in the HCI literature. We designed a set of experiments to identify regularities in 1D unidirectional moving target selection, and found a Ternary-Gaussian model to be descriptive of the endpoint distribution in such tasks. The shape of the distribution as characterized by μ and σ in the Gaussian model were primarily determined by the speed and size of the moving target. The model fits the empirical data well with 0.95 and 0.94 R2 values for μ and σ , respectively. We also demonstrated two extensions of the model, including 1) predicting error rates in moving target selection; and 2) a novel interaction technique to implicitly aid moving target selection. By applying them in a game interface design, we observed good performances in both predicting error rates (e.g., 2.7% mean absolute error) and assisting moving target selection (e.g., 33% or a greater increase in pointing accuracy).