We present the Finger-Mounted Extending Rod, a wearable device that transforms fingers into virtual tools by modulating fingertip mass distribution. We employ linear actuators on fingers that extend or retract metal rods according to their poses, generating rotational inertia while redirecting the hand to natural grip postures. Through three user studies, we evaluate (1) finger pose embodiment under visual redirection and tool matching via inertia tensor similarity, (2) perception of tool length and rotational inertia, and (3) VR tool interaction experience. Results show that 10 of 15 finger poses maintained embodiment, exhibiting inertia tensor similarities of 0.936–0.991 with their matched tools and yielding perceived inertia amplifications of 4.19–10.45×; moreover, aligning inertia tensors to virtual tools enhanced immersion, realism, and enjoyment compared to misaligned or no-device conditions across six VR scenarios. We conclude by discussing how the system renders virtual tools through the fingers and enhances their perception with inertia modulation.

Accessing real-world objects during immersive virtual reality (VR) experiences remains challenging, as current cross-reality systems often rely on predefined interaction steps, tracking devices/markers, or fixed object setups. They also lack support for personalized object recall, where users can add, remove, or modify real-world items blended into the virtual environment (VE). Many head-mounted displays (HMDs) include passthrough technology to switch between virtual and real worlds, but it often disrupts immersion by requiring a full shift from virtual to real. Thus, maintaining an optimal balance between virtuality and reality is difficult. To address these challenges, we developed SelfBlending, a framework that uses AI-based hand tracking to let users label physical objects through freehand gestures, then blends the selected item into the VE using object recognition, enabling interaction with the relevant real-world object. SelfBlending was evaluated against two common interaction conditions: the default passthrough feature in VR HMDs and the conventional approach of physically removing the HMD to access real-world objects. Results from seated, single-object interactions with tabletop-placed items showed that SelfBlending enhanced user experience by boosting presence, supporting efficient physical interaction, and improving cross-reality continuity. It also enabled selective interaction with real objects while minimizing the disruption of VR experience.

CHI ’25, Yokohama, Japan

In this paper, we present an adaptive walker system designed to address limitations in current intelligent walker technologies. While recent advancements have been made in this field, existing systems often struggle to seamlessly interpret user intent for speed control and lack adaptability across diverse scenarios and terrain. Our proposed solution incorporates high-resolution tactile sensors, deep learning algorithms, IMU sensors, and linear motors to dynamically adjust to the user's intentions and terrain changes. The system is capable of predicting the user's desired speed with an error margin of only 20.99%, relying solely on tactile input from hand and arm contact points. Additionally, it maintains the walker's horizontal stability with an error of less than 1 degree by adjusting leg lengths in response to variations in ground angle. This adaptive walker enhances user safety and comfort, particularly for individuals with reduced strength or cognitive abilities, and offers reliable assistance on uneven terrain such as uphill and downhill paths.

In-car VR applications typically synchronize virtual motion with real vehicle movement to minimize visual-vestibular mismatch. However, this approach limits virtual movement to directions in which the vehicle can physically move, typically restricting the experience to horizontal motion. This study introduces a method to expand the range of virtual motion by simulating vertical movement, leveraging vehicle acceleration to induce a vertical pitch illusion via manipulation of gravitoinertial perception. We conducted a two-phase study evaluating (1) optimal vertical gain values for maximizing perceptual realism in a controlled environment and (2) user experience factors such as motion sickness and presence in an on-road VR flight simulation under realistic driving conditions. Our findings show that users tend to prefer vertical gains that exceed theoretically valid mappings, and highlight the importance of aligning virtual motion with perceived inertial cues to enhance the realism and coherence of vertical motion in in-car VR applications.

Passengers can engage more in nondriving-related tasks owing to recent advancements in autonomous vehicles (AVs), making immersive tools such as virtual reality (VR) appealing; however, motion sickness (MS) remains a significant challenge. We present SYNC-VR, a system that aligns with visual, haptic, and auditory cues and provides proprioceptive feedback to illustrate its effect on MS and presence within the in-vehicle VR. We conducted an experiment with 24 participants using a real vehicle along a route with known MS-triggering events. Using subjective and physiological measures, we assessed participants’ presence and MS under four conditions by gradually varying the level of synchronized input sensations. Results reveal that SYNC-VR reduces MS and increases the sense of presence. Additionally, it emphasizes the impact of our interactive VR content and its role in achieving proprioceptive feedback with haptic feedback through electrical muscle stimulation, introducing an innovative approach to MS mitigation in in-vehicle VR.

This study presents ErgoPulse, a system that integrates biomechanical simulation with electrical muscle stimulation (EMS) to provide kinesthetic force feedback to the lower-body in virtual reality (VR). ErgoPulse features two main parts: a biomechanical simulation part that calculates the lower-body joint torques to replicate forces from VR environments, and an EMS part that translates torques into muscle stimulations. In the first experiment, we assessed users’ ability to discern haptic force intensity and direction, and observed variations in perceived resolution based on force direction. The second experiment evaluated ErgoPulse’s ability to increase haptic force accuracy and user presence in both continuous and impulse force VR game environments. The experimental results showed that ErgoPulse’s biomechanical simulation increased the accuracy of force delivery compared to traditional EMS, enhancing the overall user presence. Furthermore, the interviews proposed improvements to the haptic experience by integrating additional stimuli such as temperature, skin stretch, and impact.

The aesthetic design of 3D scenes in game content enhances players’ experience by inducing desired emotions. Creating emotionally engaging scenes involves designing low-level features, such as color distribution, contrast, and brightness. This study presents LumiMood, an AI-driven creativity support tool (CST) that automatically adjusts lighting and post-processing to create moods for 3D scenes. LumiMood supports designers by synthesizing reference images, creating mood templates, and providing intermediate design steps. Our formative study with 10 designers identified distinct challenges in mood design based on the participants’ experience levels. A user study involving 40 designers revealed that using LumiMood benefits the designers by streamlining workflow, improving precision, and increasing mood intention accuracy. Results indicate that LumiMood supports clarifying mood concepts and improves interpretation of lighting and post-processing, thus resolving the challenges. We observe the effect of template based designing and discuss considerable factors for AI-driven CSTs for users with varying levels of experiences.

This study introduces "Flip-Pelt," a motor-driven peltier device designed to provide rapid thermal stimulation and congruent pressure feedback in virtual reality (VR) environments. Our system incorporates eight motor-driven peltier elements, allowing for the flipping of preheated or cooled elements to the opposite side. In evaluating the Flip-Pelt device, we assess user ability to distinguish between heat/cold sources by their patterns and stiffness, and its impact on enhancing haptic experiences in VR content that involves contact with various thermal sources. Our findings demonstrate that rapid thermal stimulation and congruent pressure feedback provided by Flip-Pelt enhance the recognition accuracy of thermal patterns and the stiffness of virtual objects. These features also improve haptic experiences in VR scenarios through their temporal congruency between tactile and thermal stimuli. Additionally, we discuss the scalability of the Flip-Pelt system to other body parts by proposing design prototypes.

With the introduction of autonomous vehicles, pedestrians may no longer expect explicit communication from drivers. Despite the anticipated safety benefits of anthropomorphic human–machine interfaces (HMIs) for pedestrian crossings, the impact of different levels of anthropomorphism and communication on pedestrian safety remains insufficiently understood. We proposed a virtual-agent (VA) HMI that mimics driver behavior and investigated pedestrians’ preferences through augmented reality (AR) experiments. Eighteen participants made decisions about crossing after receiving cues about the vehicle’s intentions from VA HMIs. Participants preferred the "characterized" VA HMI owing to its aesthetically pleasing design and found the "eye contact + hand gesture" combination to be more easily comprehensible. We found that while the degree of anthropomorphism did not significantly affect pedestrians’ crossing decisions, more explicit communication was helpful. Our study provides empirical evidence regarding users’ experiences of HMI in AR and the effectiveness of VA HMIs that imitate driver communication modes.

Redirected walking (RDW) is a technology that enables users to walk seamlessly in an enormous virtual space within a narrow real space while avoiding collisions with physical elements. Although RDW provides accurate proprioceptive sensations, redirection performance is limited by visual–vestibular inconsistencies. This study aims to support seamless walking in a VR environment by alleviating inconsistencies using four vestibular stimulations: noisy and directional galvanic vestibular stimulation, bone-conduction vibration, and caloric vestibular stimulation. The user study demonstrated that the stimulations successfully enable spatial expansion without impairing immersion and presence. Non-electrical stimulations (bone-conduction vibration and caloric vestibular stimulation) expanded the detection threshold, making them alternatives to electrical stimulations, and direction-based stimulation (directional galvanic vestibular stimulation) improved the user’s gait stability in RDW. Finally, the findings suggested improving the user experience for vestibular stimulation RDW either by lowering audio interference or increasing the synchronization between the RDW gain and the stimulation intensity.

This study explored bone-conduction vibration (BCV) in redirected walking (RDW), a technology for seamless walking in large virtual spaces within confined physical areas, enhancing obstacle avoidance performance using nonelectrical vestibular stimulation without the side effects caused by electrical stimulation. We proposed four different BCV stimulation methods and evaluated their detection threshold (DT) extension performance and user experience in virtual reality (VR) conditions. The DT was successfully expanded from at least 23% to 45% under all BCV conditions while preserving the immersion and presence. Notably, user comfort increased when content sound was used for vestibular stimulation. Under the extended DT condition, a simulation study demonstrated that all BCV stimulation methods facilitated uninterrupted walking over extended distances when applying RDW to users with random movements. Thus, this research established the viability of using BCV in RDW applications and the potential for incorporating content sound into BCV stimulation techniques.

Surreal experience in virtual reality (VR) occurs when visual experience is accompanied by congruent somatosensation. Thus, VR contents that require physical actions are often bounded to our physical capabilities to maintain somatosensory consistency. Alternatively, users often choose less immersive but safer interfaces that offer a wider action variability. In either case, this situation compromises the potential for a hyper-realistic experience. To address this, we introduce “Giant Finger,” a concept that replicates human lower body movements through two enlarged virtual fingers in VR. Through a user study, we affirmed Giant Finger’s ownership using proprioceptive drift and questionnaire responses. We also compared Giant Finger’s capability to perform a variety of tasks with existing methods. Despite its minimalistic approach, Giant Finger demonstrated a high level of efficacy in supporting lower body movements, with ownership and presence comparable to those of the body-leaning method with whole-body motion. Giant Finger can replace the sensations of real legs or support locomotion in confined spaces by providing proprioceptive illusions to the virtual lower body. The applications showcased in this paper suggest that Giant Finger can enable new forms of movement with high action variability and immersion in various fields such as gaming, industry, and accessibility.

As self-driving car technology matures, autonomous vehicle research is moving toward building more human-centric interfaces and accountable experiences. Driving simulators avoid many ethical and regulatory concerns about self-driving cars and play a key role in testing new interfaces or autonomous driving scenarios. However, apart from validity studies for manual driving simulation, the capabilities of driving simulators in replicating the experience of self-driving cars have not been widely investigated. In this paper, we build six self-driving simulation platforms with varying levels of visual and motion fidelities ranging from a screen-based in-lab simulator to the mixed-reality on-road simulator we propose. We compare the sense of presence and simulator sickness for each simulator composition, as well as its visual and motion fidelities with a user study. Our novel mixed-reality automotive driving simulator, named MAXIM, showed highest fidelity and presence. Our findings suggest how visual and motion configurations affect experience in autonomous driving simulators.

The recent profusion of sensors has given consumers and researchers the ability to collect significant amounts of data. However, understanding sensor data can be a challenge, because it is voluminous, multi-sourced, and unintelligible. Nonetheless, intelligent systems, such as activity recognition, require pattern analysis of sensor data streams to produce compelling results; machine learning (ML) applications enable this type of analysis. However, the number of ML experts able to proficiently classify sensor data is limited, and there remains a lack of interactive, usable tools to help intermediate users perform this type of analysis. To learn which features these tools must support, we conducted interviews with intermediate users of ML and conducted two probe-based studies with a prototype ML and visual analytics system, Gimlets. Our system implements ML applications for sensor-based time-series data as a novel domain-specific prototype that integrates interactive visual analytic features into the ML pipeline. We identify future directions for usable ML systems based on sensor data that will enable intermediate users to build systems that have been prohibitively difficult.

Interruptions while driving can be quite dangerous, whether these are self-interruptions or external interruptions. They increase driver workload and reduce performance on the primary driving task. Being able to identify when a driver is interruptible is critical for building systems that can mediate these interruptions. In this paper, we collect sensor and human-annotated data from 15 drivers, including vehicle motion, traffic states, physiological responses and driver motion. We demonstrate that this data can be used to build a machine learning classifier that can determine interruptibility every second with a 94% accuracy. We present both population and individual models and discuss the features that contribute to the high performance of this system. Such a classifier can be used to build systems that mediate when drivers use technology to self-interrupt and when drivers are interrupted by technology.

The elder population is rising worldwide; in the US, no longer being able to drive is a significant marker of loss of independence. One of the approaches to helping elders drive more safely is to investigate the use of automotive user interface technology, and specifically, to explore the instrument panel (IP) display design to help attract and manage attention and make information easier to interpret.

With a focus on presenting information at the right time, the ubicomp community can benefit greatly from learning the most salient human measures of cognitive load. Cognitive load can be used as a metric to determine when or whether to interrupt a user. In this paper, we collected data from multiple sensors and compared their ability to assess cognitive load. Our focus is on visual perception and cognitive speed-focused tasks that leverage cognitive abilities common in ubicomp applications. We found that across all participants, the electrocardiogram median absolute deviation and median heat flux measurements were the most accurate at distinguishing between low and high levels of cognitive load, providing a classification accuracy of over 80% when used together. Our contribution is a real-time, objective, and generalizable method for assessing cognitive load in cognitive tasks commonly found in ubicomp systems and situations of divided attention.

A common effect of aging is decline in spatial cognition. This is an issue for all elders, but particularly for elder drivers. To address this driving issue, we propose a novel concept of an in-vehicle navigation display system that displays navigation information directly onto the vehicle's windshield, superimposing it on the driver's view of the actual road. An evaluation of our simulated version of this display shows that it results in a significant reduction in navigation errors and distraction-related measures compared to a typical in-car navigation display for elder drivers. These results help us understand how context-sensitive information and a simulated augmented reality representation can be combined to minimize the cognitive load in translating between virtual/information spaces and the real world.