On-body computing systems offer new forms of interaction, but while they are increasingly integrated into everyday contexts, their unique privacy and safety challenges remain understudied. This paper examines these challenges through a two-round interview study with N = 15 experts in human-computer interaction, and privacy and safety, using speculative scenarios and adversarial roleplaying to elicit insights. Our findings reveal risks specific to on-body interactions, including over-collection of sensitive data, unwanted inferences, harm to bystanders, and threats to bodily autonomy and psychological well-being. Importantly, in the on-body context, privacy and safety concerns are deeply interconnected and cannot be addressed in isolation. We contribute an empirically grounded characterization of these entangled challenges and derive eight actionable design guidelines to support safer, more privacy-aware, on-body systems. This work informs future research and design in ubiquitous computing by highlighting the need for proactive and integrated approaches to privacy and safety in trustworthy on-body computing.

In virtual reality (VR) experiences, mismatches between reality and virtuality are usually undesirable, as they can disrupt immersion and induce cybersickness. However, when carefully controlled, they may expand the design space of VR. This research investigates perceptual detection thresholds for mismatches between real and virtual inclinations during cycling in VR. Using a custom simulation, N = 30 participants cycled through a virtual city while physical and visual inclinations were independently manipulated. Real inclinations were implemented with a tilting indoor bike, providing vestibular and proprioceptive feedback, while virtual inclinations within the simulated environment were presented visually. A multiple staircase procedure derived estimates for perceptual thresholds that approximate which mismatches in visual and physical inclination were still perceived as congruent. These thresholds reveal a window of perceived congruence before mismatches become noticeable to users. These findings advance understanding of sensory integration in VR cycling and inform applications in immersive training, exergames, and rehabilitation systems.

Figure 1: We present a bimanual motion-coupled vibration algorithm capable of creating connected material experiences between two hands.By modulating the parameters of vibrations between the hands, our algorithm can induce material properties of elasticity while stretching A ○, flexibility while bending B○ and torsion while twisting C ○.

The way users hold a smartphone depends on the interaction task, yet little is known about the fingers’ engagement with the device’s surfaces beyond the touchscreen. Such an understanding not only opens up opportunities for novel on- and off-screen interactions, but also the device’s possible physical affordances. We present a study (N=23) that examines the hands’ physical engagement with the smartphone beyond the touchscreen across nine mobile interactions. Grasps were annotated from photographs, and contact regions were captured using residual heat traces from grasping the device. Our findings show that fingers and palms adopt a variety of support roles and postures when engaging with the smartphone’s back and side edges. The hand-contact maps reveal distinct patterns, differing in contact frequency and placement. This work contributes an empirical characterisation of hands’ back and edge engagement, highlighting design opportunities for future smartphone usage extending beyond the touchscreen.

Space agencies are in the process of drawing up carefully thought-out Concepts of Operations (ConOps) for future human missions on the Moon. These are typically assessed and validated through costly and logistically demanding analogue field studies. While interactive simulations in Virtual Reality (VR) offer a comparatively cost-effective alternative, they have faced criticism for lacking the fidelity of real-world deployments. This paper explores the applicability of passive haptic interfaces in bridging the gap between simulated and real-world ConOps assessments. Leveraging passive haptic props (equipment mockup and astronaut gloves), we virtually recreated the Apollo 12 mission procedure and assessed it with experienced astronauts and other space experts. Quantitative and qualitative findings indicate that haptics increased presence and embodiment, thus improving perceived simulation fidelity and validity of user reflections. We conclude by discussing the potential role of passive haptic modalities in facilitating early-stage ConOps assessments for human endeavours on the Moon and beyond.

To enhance interactions in VR, many techniques introduce offsets between the virtual and real-world position of users’ hands. Nevertheless, such hand redirection (HR) techniques are only effective as long as they go unnoticed by users—not disrupting the VR experience. While several studies consider how much unnoticeable redirection can be applied, these focus on mid-air floating hands that are disconnected from users’ bodies. Increasingly, VR avatars are embodied as being directly connected with the user’s body, which provide more visual cue anchoring, and may therefore reduce the unnoticeable redirection threshold. In this work, we studied more complete avatars and their effect on the sense of embodiment and the detectability of HR. We found that higher avatar completeness increases embodiment, and we provide evidence for the absence of practically relevant effects on the detectability of HR.

In pursuit of hand redirection techniques that are ever more tailored to human perception, we propose the first algorithm for hand redirection in virtual reality that makes use of saccades, i.e., fast ballistic eye movements that are accompanied by the perceptual phenomenon of change blindness. Our technique combines the previously proposed approaches of gradual hand warping and blink-suppressed hand redirection with the novel approach of saccadic redirection in one unified yet simple algorithm. We compare three variants of the proposed Saccadic & Blink-Suppressed Hand Redirection (SBHR) technique with the conventional approach to redirection in a psychophysical study (N = 25). Our results highlight the great potential of our proposed technique for comfortable redirection by showing that SBHR allows for significantly greater magnitudes of unnoticeable redirection while being perceived as significantly less intrusive and less noticeable than commonly employed techniques that only use gradual hand warping.

Many illusion and interaction techniques in Virtual Reality (VR) rely on Hand Redirection (HR), which has proved to be effective as long as the introduced offsets between the position of the real and virtual hand do not noticeably disturb the user experience. Yet calibrating HR offsets is a tedious and time-consuming process involving psychophysical experimentation, and the resulting thresholds are known to be affected by many variables—limiting HR’s practical utility. As a result, there is a clear need for alternative methods that allow tailoring HR to the perceptual boundaries of individual users. We conducted an experiment with 18 participants combining movement, eye gaze and EEG data to detect HR offsets Below, At, and Above individuals’ detection thresholds. Our results suggest that we can distinguish HR At and Above from no HR. Our exploration provides a promising new direction with potentially strong implications for the broad field of VR illusions.

As more and more virtual reality (VR) headsets support eye tracking, recent techniques started to use eye blinks to induce unnoticeable manipulations to the virtual environment, e.g., to redirect users’ actions. However, to exploit their full potential, more control over users’ blinking behavior in VR is required. To this end, we propose a set of reflex-based blink triggers that are suited specifically for VR. In accordance with blink-based techniques for redirection, we formulate (i) effectiveness, (ii) efficiency, (iii) reliability, and (iv) unobtrusiveness as central requirements for successful triggers. We implement the soft- and hardware-based methods and compare the four most promising approaches in a user study. Our results highlight the pros and cons of the tested triggers, and show those based on the menace, corneal, and dazzle reflexes to perform best. From these results, we derive recommendations that help choosing suitable blink triggers for VR applications.

Rendering haptic feedback for interactions with virtual objects is an essential part of effective virtual reality experiences. In this work, we explore providing haptic feedback for rotational manipulations, e.g., through knobs. We propose the use of a Pseudo-Haptic technique alongside a physical proxy knob to simulate various physical resistances. In a psychophysical experiment with 20 participants, we found that designers can introduce unnoticeable offsets between real and virtual rotations of the knob, and we report the corresponding detection thresholds. Based on these, we present the Pseudo-Haptic Resistance technique to convey physical resistance while applying only unnoticeable pseudo-haptic manipulation. Additionally, we provide a first model of how C/D gains correspond to physical resistance perceived during object rotation, and outline how our results can be translated to other rotational manipulations. Finally, we present two example use cases that demonstrate the versatility and power of our approach.

Handheld virtual reality (VR) controllers enable users to manipulate virtual objects in VR but do not convey a virtual object's weight. This hinders users from effectively experiencing lighter and heavier objects. While previous work explored either hardware-based interfaces or software-based pseudo-haptics, in this paper, we combine two techniques to improve the virtual weight perception in VR. By adapting the trigger resistance of the VR controller when grasping a virtual object and manipulating the control-display (C/D) ratio during lifting, we create a continuous weight sensation. In a psychophysical study (N=29), we compared our combined approach against the individual rendering techniques. Our results show that participants were significantly more sensitive towards smaller weight differences in the combined weight simulations compared to the individual methods. Additionally, participants were also able to determine weight differences significantly faster with both cues present compared to the single pseudo-haptic technique. While all three techniques were generally valued to be effective, the combined approach was favoured the most. Our findings demonstrate the meaningful benefit of combining physical and virtual techniques for virtual weight rendering over previously proposed methods.

While consumer-grade virtual reality (VR) hardware can deliver immersive audiovisual experiences, these systems often lack the ability to display realistic haptic feedback, or incorporate cost-efficient vibrotactile actuators with very limited abilities to provide tactile feedback. To overcome these limitations, we introduce an approach based on audio-based vibrotactile actuators. Due to their wide frequency response and multiple resonant frequencies, they can provide more tactile details. In our implementation, every VR interaction uses standard audio clips to provide simultaneous auditory and tactile feedback, as well as coupled realistic physics simulations for the visual feedback. We evaluate our approach to assess the benefits on the user’s experience regarding various interaction scenarios in VR, comparing our approach to a simulated fixed-frequency actuator as a baseline. The results confirmed the benefits of our approach in terms of user preference, perceived realism, comfort, sense of agency, and texture perception. Furthermore, multimodal feedback resulted in the best user experience.

On the way towards novel hand redirection (HR) techniques that make use of change blindness, the next step is to take advantage of saccades for hiding body warping. A prerequisite for saccadic HR algorithms, however, is to know how much the user’s virtual hand can unnoticeably be offset during saccadic suppression. We contribute this knowledge by conducting a psychophysical experiment, which lays the ground for upcoming HR techniques by exploring the perceptual detection thresholds (DTs) of hand offset injected during saccades. Our findings highlight the pivotal role of saccade direction for unnoticeable hand jumps, and reveal that most offset goes unnoticed when the saccade and hand move in opposite directions. Based on the gathered perceptual data, we derived a model that considers the angle between saccade and hand offset direction to predict the DTs of saccadic hand jumps.

We propose a novel open-source software toolkit to support researchers in the domains of human-computer interaction (HCI) and virtual reality (VR) in conducting psychophysical experiments. Our toolkit is designed to work with the widely-used Unity engine and is implemented in C# and Python. With the toolkit, researchers can easily set up, run, and analyze experiments to find perceptual detection thresholds using the adaptive weighted up/down method, also known as the staircase procedure. Besides being straightforward to integrate in Unity projects, the toolkit automatically stores experiment results, features a live plotter that visualizes answers in real time, and offers scripts that help researchers analyze the gathered data using statistical tests.

Providing haptic feedback when manipulating virtual objects is an essential part of immersive virtual reality experiences; however, it is challenging to replicate all of an object's properties and characteristics. We propose the use of visuo-haptic illusions alongside physical proxies to enhance the scope of proxy-based interactions with virtual objects. In this work, we focus on two manipulation techniques, linear translation and stretching across different distances, and investigate how much discrepancy between the physical proxy and the virtual object may be introduced without participants noticing. In a study with 24 participants, we found that manipulation technique and travel distance significantly affect the detection thresholds, and that visuo-haptic illusions impact performance and accuracy. We show that this technique can be used to enable functional proxy objects that act as stand-ins for multiple virtual objects, illustrating the technique through a showcase VR-DJ application.

Using virtual reality setups, users can fade out of their surroundings and dive fully into a thrilling and appealing virtual environment. The success of such immersive experiences depends heavily on natural and engaging interactions with the virtual world. As developers tend to focus on intuitive hand controls, other aspects of the broad range of full-body capabilities are easily left vacant. One repeatedly overlooked input modality is the user's gait. Even though users may walk physically to explore the environment, it usually does not matter how they move. However, gait-based interactions, using the variety of information contained in human gait, could offer interesting benefits for immersive experiences. For instance, stealth VR-games could profit from this additional range of interaction fidelity in the form of a sneaking-based input modality. In our work, we explore the potential of sneaking as a playful input modality for virtual environments. Therefore, we discuss possible sneaking-based gameplay mechanisms and develop three technical approaches, including precise foot-tracking and two abstraction levels. Our evaluation reveals the potential of sneaking-based inter-actions in IVEs, offering unique challenges and thrilling gameplay. For these interactions, precise tracking of individual footsteps is unnecessary, as a more abstract approach focusing on the players' intention offers the same experience while providing better comprehensible feedback. Based on these findings, we discuss the broader potential and individual strengths of our gait-centered interactions.

Many interaction techniques in virtual reality break with the 1-to-1 mapping from real to virtual space. Instead, specialized techniques for 3D interaction and haptic retargeting leverage hand redirection, offsetting the virtual hand rendering from the real hand position. To achieve unnoticeable hand redirection, however, the utilization of change blindness phenomena has not been systematically explored. Inspired by recent advances in the domain of redirected walking, we present the first hand redirection technique that makes use of blink-induced visual suppression and corresponding change blindness. We introduce Blink-Suppressed Hand Redirection (BSHR) to study the feasibility and detectability of hand redirection based on blink suppression. Our technique is based on Cheng et al.'s (2017) [9] body warping algorithm and instantaneously shifts the virtual hand when the user's vision is suppressed during a blink. Additionally, it can be configured to continuously increment hand offsets when the user's eyes are opened, limited to an extent below detection thresholds. In a psychophysical experiment, we verify that unnoticeable blink-suppressed hand redirection is possible even in worst -case scenarios, and derive the corresponding conservative detection thresholds (CDTs). Moreover, our results show that the range of unnoticeable redirection can be increased by combining continuous warping and blink-suppressed instantaneous shifts. As an additional contribution, we derive the CDTs for Cheng et al.'s (2017) [9] redirection technique that does not leverage blinks.

To provide immersive haptic experiences, proxy-based haptic feedback systems for virtual reality (VR) face two central challenges: (1) similarity, and (2) colocation. While to solve challenge (1), physical proxy objects need to be sufficiently similar to their virtual counterparts in terms of haptic properties, for challenge (2), proxies and virtual counterparts need to be sufficiently colocated to allow for seamless interactions. To solve these challenges, past research introduced, among others, two successful techniques: (a) Dynamic Passive Haptic Feedback (DPHF), a hardware-based technique that leverages actuated props adapting their physical state during the VR experience, and (b) Haptic Retargeting, a software-based technique leveraging hand redirection to bridge spatial offsets between real and virtual objects. Both concepts have, up to now, not ever been studied in combination. This paper proposes to combine both techniques and reports on the results of a perceptual and a psychophysical experiment situated in a proof-of-concept scenario focused on the perception of virtual weight distribution. We show that users in VR overestimate weight shifts and that, when DPHF and HR are combined, significantly greater shifts can be rendered, compared to using only a weight-shifting prop or unnoticeable hand redirection. Moreover, we find the combination of DPHF and HR to let significantly larger spatial dislocations of proxy and virtual counterpart go unnoticed by users. Our investigation is the first to show the value of combining DPHF and HR in practice, validating that their combination can better solve the challenges of similarity and colocation than the individual techniques can do alone.

Manipulating the appearance of a Virtual Environment to enable natural walking has so far focused on modifications that are intended to be unnoticed by users. In our research, we took a radically different approach by embracing the overt nature of the change. To explore this method, we designed the Space Bender, a natural walking technique for room-scale VR. It builds on the idea of overtly manipulating the Virtual Environment by "bending" the geometry whenever the user comes in proximity of a physical boundary. Our aim was to evaluate the feasibility of this approach in terms of performance and subjective feedback. We compared the Space Bender to two other similarly situated techniques: Stop and Reset and Teleportation, in a task requiring participants to traverse a 100 m path. Results show that the Space Bender was significantly faster than Stop and Reset, and preferred to the Teleportation technique, highlighting the potential of overt manipulation to facilitate natural walking.

In many professional domains, relevant processes are documented as abstract process models, such as event-driven process chains (EPCs). EPCs are traditionally visualized as 2D graphs and their size varies with the complexity of the process. While process modeling experts are used to interpreting complex 2D EPCs, in certain scenarios such as, for example, professional training or education, also novice users inexperienced in interpreting 2D EPC data are facing the challenge of learning and understanding complex process models. To communicate process knowledge in an effective yet motivating and interesting way, we propose a novel virtual reality (VR) interface for non-expert users. Our proposed system turns the exploration of arbitrarily complex EPCs into an interactive and multi-sensory VR experience. It automatically generates a virtual 3D environment from a process model and lets users explore processes through a combination of natural walking and teleportation. Our immersive interface leverages basic gamification in the form of a logical walkthrough mode to motivate users to interact with the virtual process. The generated user experience is entirely novel in the field of immersive data exploration and supported by a combination of visual, auditory, vibrotactile and passive haptic feedback. In a user study with N=27 novice users, we evaluate the effect of our proposed system on process model understandability and user experience, while comparing it to a traditional 2D interface on a tablet device. The results indicate a tradeoff between efficiency and user interest as assessed by the UEQ novelty subscale, while no significant decrease in model understanding performance was found using the proposed VR interface. Our investigation highlights the potential of multi-sensory VR for less time-critical professional application domains, such as employee training, communication, education, and related scenarios focusing on user interest.

Experiencing materials in virtual reality (VR) is enhanced by combining visual and haptic feedback. While VR easily allows changes to visual appearances, modifying haptic impressions remains challenging. Existing passive haptic techniques require access to a large set of tangible proxies. To reduce the number of physical representations, we look towards fabrication to create more versatile counterparts. In a user study, 3D-printed hairs with length varying in steps of 2.5 mm were used to influence the feeling of roughness and hardness. By overlaying fabricated hair with visual textures, the resolution of the user's haptic perception increased. As changing haptic sensations are able to elicit perceptual switches, our approach can extend a limited set of textures to a much broader set of material impressions. Our results give insights into the effectiveness of 3D-printed hair for enhancing texture perception in VR.

Standard controllers for virtual reality (VR) lack sophisticated means to convey a realistic, kinesthetic impression of size, resistance or inertia. We present the concept and implementation of Drag:on, an ungrounded shape-changing VR controller that provides dynamic passive haptic feedback based on drag, i.e. air resistance, and weight shift. Drag:on leverages the airflow occurring at the controller during interaction. By dynamically adjusting its surface area, the controller changes the drag and rotational inertia felt by the user. In a user study, we found that Drag:on can provide distinguishable levels of haptic feedback. Our prototype increases the haptic realism in VR compared to standard controllers and when rotated or swung improves the perception of virtual resistance. By this, Drag:on provides haptic feedback suitable for rendering different virtual mechanical resistances, virtual gas streams, and virtual objects differing in scale, material and fill state.

Virtual reality (VR) interaction techniques like haptic retargeting offset the user's rendered virtual hand from the real hand location to redirect the user's physical hand movement. This paper explores the order of magnitude of hand redirection that can be applied without the user noticing it. By deriving lower-bound estimates of detection thresholds, we quantify the range of unnoticeable redirection for the three basic redirection dimensions, horizontal, vertical and gain-based hand warping. In a two-alternative forced choice (2AFC) experiment, we individually explore these three hand warping dimensions each in three different scenarios: a very conservative scenario without any distraction and two conservative but more realistic scenarios that distract users from the redirection. Additionally, we combine the results of all scenarios to derive robust recommendations for each redirection technique. Our results indicate that within a certain range, desktop-scale VR hand redirection can go unnoticed by the user, but that this range is narrow. The findings show that the virtual hand can be unnoticeably displaced horizontally or vertically by up to 4.5° in either direction, respectively. This allows for a range of ca. 9°, in which users cannot reliably detect applied redirection. For our gain-based hand redirection technique, we found that gain factors between g = 0.88 and g = 1.07 can go unnoticed, which corresponds to a user grasping up to 13.75% further or up to 6.18% less far than in virtual space. Our findings are of value for the development of VR applications that aim to redirect users in an undetectable manner, such as for haptic retargeting.

We define the concept of Dynamic Passive Haptic Feedback (DPHF) for virtual reality by introducing the weight-shifting physical DPHF proxy object Shifty. This concept combines actuators known from active haptics and physical proxies known from passive haptics to construct proxies that automatically adapt their passive haptic feedback. We describe the concept behind our ungrounded weight-shifting DPHF proxy Shifty and the implementation of our prototype. We then investigate how Shifty can, by automatically changing its internal weight distribution, enhance the user's perception of virtual objects interacted with in two experiments. In a first experiment, we show that Shifty can enhance the perception of virtual objects changing in shape, especially in length and thickness. Here, Shifty was shown to increase the user's fun and perceived realism significantly, compared to an equivalent passive haptic proxy. In a second experiment, Shifty is used to pick up virtual objects of different virtual weights. The results show that Shifty enhances the perception of weight and thus the perceived realism by adapting its kinesthetic feedback to the picked-up virtual object. In the same experiment, we additionally show that specific combinations of haptic, visual and auditory feedback during the pick-up interaction help to compensate for visual-haptic mismatch perceived during the shifting process.