Research in Augmented Reality (AR) and Virtual Reality (VR) has mostly viewed them in isolation.Yet, when used together in practical settings, AR and VR each offer unique strengths, necessitating multiple transitions to harness their advantages.This paper investigates potential challenges in Cross-Reality (CR) transitions to inform future application design.We implemented a CR system featuring a 3D modeling task that requires users to switch between PC, AR, and VR.Using a talk-aloud study (n=12) and thematic analysis, we revealed that frictions primarily arose when transitions conflicted with users' Spatial Mental Model (SMM).Furthermore, we found five transition archetypes employed to enhance productivity once an SMM was established.Our findings uncover that transitions have to focus on establishing and upholding the SMM of users across realities, by communicating differences between them.

User interaction with digital systems requires Fine Motor Control (FMC), especially if the interfaces are complex or require high fidelity and fine-grained interactions. Despite its importance, Fine Motor Control is often overlooked in interactive system design, partly because of its complex assessment. Measuring changes in fine motor abilities due to prolonged use or fatigue currently requires repeated manual testing. This paper analyzes the concept of using the digital mobile devices’ input behavior to assess the user’s Fine Motor Control. For this, we show that Fine Motor Control can be assessed for touch and stylus-based interaction with a digital mobile system. We conducted a user study, where participants performed a Nine Hole Peg Test and a predefined Copy Drawing Test before and after exercises that affect fine motor skills. Based on this data, we investigated how metrics such as pressure, velocity, and entropy for touch and stylus input can be used to predict Fine Motor Control.

Advancements in haptics for Virtual Reality (VR) increased the quality of immersive content. Particularly, recent efforts to provide realistic temperature sensations have gained traction, but most often require very specialized or large complex devices to create precise thermal actuations. However, being largely detached from the real world, such a precise correspondence between the physical location of thermal stimuli and the shown visuals in VR might not be necessary for an authentic experience. In this work, we contribute the findings of a controlled experiment with 20 participants, investigating the spatial localization accuracy of thermal stimuli while having matching and non-matching visual cues of a virtual heat source in VR. Although participants were highly confident in their localization decisions, their ability to accurately pinpoint thermal stimuli was notably deficient.

In recent years, the community has presented various novel solutions to address the lack of haptic feedback in virtual reality experiences. Yet, it remains a major challenge for Virtual Reality applications. Encountered-type Haptic Displays (ETHDs) have emerged as a promising alternative to enable haptic feedback in VR without requiring the user to wear any device while allowing for sensorily rich experiences such as texture, kinaesthetic feedback, and even ultrasonic tactile feedback. Nevertheless, as important as thermal feedback is for daily life interactions, such as assessing the temperature of a mug or knowing if the microwave is on, thermal feedback in ETHD has remained largely unexplored. In this paper, we present a novel ETHD that provides thermal feedback and explore its potential in VR. We describe the design of our ETHD, and we report the results of a user study that compares different thermal feedback settings in VR. Our results show that thermal feedback can significantly enhance the user immersion and haptic experience in VR, and we discuss the implications of our findings for the design of ETHD and VR experiences.

MobiTouch is a wearable haptic device that integrates pneumatic and vibrotactile technologies to enhance touch sensations. The system addresses the slow responsiveness of small pneumatic components by compensating with vibrations, allowing for more diverse touch patterns. The wrist prototype operates wireless, is powered by a rechargeable battery, and features customisable touch patterns through an external controller. MobiTouch’s design improves portability and responsiveness for haptic pneumatic wearables, offering potential applications in affective touch and wireless tactile transmission.

From carrying grocery bags to holding onto handles on the bus, there are a variety of situations where one or both hands are busy, hindering the vision of ubiquitous interaction with technology. Voice commands, as a popular hands-free alternative, struggle with ambient noise and privacy issues. As an alternative approach, research explored movements of various body parts (e.g., head, arms) as input modalities, with foot-based techniques proving particularly suitable for hands-free interaction. Whereas previous research only considered the movement of the foot as a whole, in this work, we argue that our toes offer further degrees of freedom that can be leveraged for interaction. To explore the viability of toe-based interaction, we contribute the results of a controlled experiment with 18 participants assessing the impact of five factors on the accuracy, efficiency and user experience of such interfaces. Based on the findings, we provide design recommendations for future toe-based interfaces.

Mid-air gestures, widely used in today’s Augmented Reality (AR) applications, are prone to the “gorilla arm” effect, leading to discomfort with prolonged interactions. While prior work has proposed metrics to quantify this effect and means to improve comfort and ergonomics, these works usually only consider simplistic, one-dimensional AR interactions, like reaching for a point or pushing a button. However, interacting with AR environments also involves far more complex tasks, such as rotational knobs, potentially impacting ergonomics. This paper advances the understanding of the ergonomics of rotational mid-air interactions in AR. For this, we contribute the results of a controlled experiment exposing the participants to a rotational task in the interaction space defined by their arms’ reach. Based on the results, we discuss how novel future mid-air gesture modalities benefit from our findings concerning ergonomic-aware rotational interaction.

Thermal feedback has been studied for navigation purposes with directional cues and a variety of other use cases. Yet, to date, systems providing thermal feedback were primarily designed for the upper body, targeting hands and arms in particular. As these parts are often occupied with other tasks, there is a need to extend the design space of thermal feedback to other body parts. To close this gap, we assess thermal feedback on the user’s feet. This research explores if creating stimuli representing any direction on a circle with only four actuators is possible. To evaluate this concept, we conducted a user study asking the participants to indicate the perceived direction after getting a hot or cold stimulus by direct actuation using one actuator or phantom actuation using two actuators. The results indicate that the detection accuracy was higher for cold signals. In addition, the results showed higher recognition for stimuli linked to actuator distribution than phantom sensation due to spatial summation.

Exergames, video games designed to blend entertainment with physical activity, aim to improve users’ physical fitness by combining gaming with exercise. However, integrating exercise equipment, such as rowers, bikes, and ski ergometers into Virtual Reality (VR) environments remains challenging. In this poster, we introduce a toolkit that simplifies the integration of ergometers into Unity-based projects. Researchers can access detailed ergometer data for logging and use inside their projects, while our toolkit handles tedious tasks, like connection-handling or parsing. VRow offers valuable support for creating immersive and interactive fitness experiences.

Haptic Feedback is essential for lifelike Virtual Reality (VR) experiences. To provide a wide range of matching sensations of being touched or stroked, current approaches typically need large numbers of different physical textures. However, even advanced devices can only accommodate a limited number of textures to remain wearable. Therefore, a better understanding is necessary of how expectations elicited by different visualizations affect haptic perception, to achieve a balance between physical constraints and great variety of matching physical textures.

Cycling has become increasingly popular as a means of transportation. However, cyclists remain a highly vulnerable group of road users. According to accident reports, one of the most dangerous situations for cyclists are uncontrolled intersections, where cars approach from both directions. To address this issue and assist cyclists in crossing decision-making at uncontrolled intersections, we designed two visualizations that: (1) highlight occluded cars through an X-ray vision and (2) depict the remaining time the intersection is safe to cross via a Countdown. To investigate the efficiency of these visualizations, we proposed an Augmented Reality simulation as a novel evaluation method, in which the above visualizations are represented as AR, and conducted a controlled experiment with 24 participants indoors. We found that the X-ray ensures a fast selection of shorter gaps between cars, while the Countdown facilitates a feeling of safety and provides a better intersection overview.

From zooming on smartphones and mid-air gestures to deformable user interfaces, thumb-index pinching grips are used in many interaction techniques. However, there is still a lack of systematic understanding of how the accuracy and efficiency of such grips are affected by various factors such as counterforce, grip span, and grip direction. Therefore, in this paper, we contribute an evaluation (N = 18) of thumb-index pinching performance in a visual targeting task using scales up to 75 items. As part of our findings, we conclude that the pinching interaction between the thumb and index finger is a promising modality also for one-dimensional input on higher scales. Furthermore, we discuss and outline implications for future user interfaces that benefit from pinching as an additional and complementary interaction modality.

Augmented Reality (AR) has influenced the presentation of historical information to tourists and museum visitors by making the information more immersive and engaging. Since smartphones and AR glasses are the primary devices to present AR information to users, it is essential to understand how the information about a historical site can be presented effectively and what type of device is best suited for information placements. In this paper, we investigate the placement of two types of content, historical images and informational text, for smartphones and AR glasses in the context of outdoor historical sites. For this, we explore three types of placements: (1) on-body, (2) world, and (3) overlay. To evaluate all nine combinations of text and image placements for smartphone and AR glasses, we conducted a controlled experiment (N = 18) at outdoor historical landmarks. We discovered that on-body image and text placements were the most convenient compared to overlay and world for both devices. Furthermore, participants found themselves more successful in exploring historical sites using a smartphone than AR glasses. Although interaction with a smartphone was more convenient, participants found exploring AR content using AR glasses more fun.

Computer-supported posture guidance is used in sports, dance training, expression of art with movements, and learning gestures for interaction. At present, the influence of display types and visualizations have not been investigated in the literature. These factors are important as they directly impact perception and cognitive load, and hence influence the performance of participants. In this paper, we conducted a controlled experiment with 20 participants to compare the use of five display types with different screen sizes: smartphones, tablets, desktop monitors, TVs, and large displays. On each device, we compared three common visualizations for posture guidance: skeletons, silhouettes, and 3d body models. To conduct our assessment, we developed a mobile and cross-platform system that only requires a single camera. Our results show that compared to a smartphone display, larger displays show a lower error (12%). Regarding the choice of visualization, participants rated 3D body models as significantly more usable in comparison to a skeleton visualization.

Recent technological advances have made head-mounted displays (HMDs) smaller and untethered, fostering the vision of ubiquitous interaction in a digitally augmented physical world. Consequently, a major part of the interaction with such devices will happen on the go, calling for interaction techniques that allow users to interact while walking. In this paper, we explore lateral shifts of the walking path as a hands-free input modality. The available input options are visualized as lanes on the ground parallel to the user's walking path. Users can select options by shifting the walking path sideways to the respective lane. We contribute the results of a controlled experiment with 18 participants, confirming the viability of our approach for fast, accurate, and joyful interactions. Further, based on the findings of the controlled experiment, we present three example applications.

Recent advances have made Virtual Reality (VR) more realistic than ever before. This improved realism is attributed to today's ability to increasingly appeal to human sensations, such as visual, auditory or tactile. While research also examines temperature sensation as an important aspect, the interdependency of visual and thermal perception in VR is still underexplored. In this paper, we propose Therminator, a thermal display concept that provides warm and cold on-body feedback in VR through heat conduction of flowing liquids with different temperatures. Further, we systematically evaluate the interdependency of different visual and thermal stimuli on the temperature perception of arm and abdomen with 25 participants. As part of the results, we found varying temperature perception depending on the stimuli, as well as increasing involvement of users during conditions with matching stimuli.

Abstract This paper presents a brainstorming tool combined with pointing gestures to improve the brainstorming meeting experience for blind and visually impaired people (BVIP). In brainstorming meetings, BVIPs are not able to participate in the conversation as well as sighted users because of the unavailability of supporting tools for understanding the explicit and implicit meaning of the non-verbal communication (NVC). Therefore, the proposed system assists BVIP in interpreting pointing gestures which play an important role in non-verbal communication. Our system will help BVIP to access the contents of a Metaplan card, a team member in the brainstorming meeting is referring to by pointing. The prototype of our system shows that targets on the screen a user is pointing at can be detected with 80% accuracy.

Abstract In recent years, assistive technology and digital accessibility for blind and visually impaired people (BVIP) has been significantly improved. Yet, group discussions, especially in a business context, are still challenging as non-verbal communication (NVC) is often depicted on digital whiteboards, including deictic gestures paired with visual artifacts. However, as NVC heavily relies on the visual perception, whichrepresents a large amount of detail, an adaptive approach is required that identifies the most relevant information for BVIP. Additionally, visual artifacts usually rely on spatial properties such as position, orientation, and dimensions to convey essential information such as hierarchy, cohesion, and importance that is often not accessible to the BVIP. In this paper, we investigate the requirements of BVIP during brainstorming sessions and, based on our findings, provide an accessible multimodal tool that uses non-verbal and spatial cues as an additional layer of information. Further, we contribute by presenting a set of input and output modalities that encode and decode information with respect to the individual demands of BVIP and the requirements of different use cases.

Virtual Reality Environments (VRE) create an immersive user experience through visual, aural, and haptic sensations. However, the latter is often limited to vibrotactile sensations that are not able to actively provide kinesthetic motion actuation. Further, such sensations do not cover natural representations of physical forces, for example, when lifting a weight. We present PneumAct, a jacket to enable pneumatically actuated kinesthetic movements of arm joints in VRE. It integrates two types of actuators inflated through compressed air: a Contraction Actuator and an Extension Actuator. We evaluate our PneumAct jacket through two user studies with a total of 32 participants: First, we perform a technical evaluation measuring the contraction and extension angles of different inflation patterns and inflation durations. Second, we evaluate PneumAct in three VRE scenarios comparing our system to traditional controller-based vibrotactile and a baseline without haptic feedback.

From voice commands and air taps to touch gestures on frames: Various techniques for interacting with head-mounted displays (HMDs) have been proposed. While these techniques have both benefits and drawbacks dependent on the current situation of the user, research on interacting with HMDs has not concluded yet. In this paper, we add to the body of research on interacting with HMDs by exploring foot-tapping as an input modality. Through two controlled experiments with a total of 36 participants, we first explore direct interaction with interfaces that are displayed on the floor and require the user to look down to interact. Secondly, we investigate indirect interaction with interfaces that, although operated by the user's feet, are always visible as they are floating in front of the user. Based on the results of the two experiments, we provide design recommendations for direct and indirect foot-based user interfaces.

Room-scale Virtual Reality (VR) systems have arrived in users' homes where tracked environments are set up in limited physical spaces. As most Virtual Environments (VEs) are larger than the tracked physical space, locomotion techniques are used to navigate in VEs. Currently, in recent VR games, point & teleport is the most popular locomotion technique. However, it only allows users to select the position of the teleportation and not the orientation that the user is facing after the teleport. This results in users having to manually correct their orientation after teleporting and possibly getting entangled by the cable of the headset. In this paper, we introduce and evaluate three different point & teleport techniques that enable users to specify the target orientation while teleporting. The results show that, although the three teleportation techniques with orientation indication increase the average teleportation time, they lead to a decreased need for correcting the orientation after teleportation.

Embedding sensors into objects allow them to recognize various interactions. However, sensing usually requires active electronics that are often costly, need time to be assembled, and constantly draw power. Thus, we propose off-line sensing: passive 3D-printed sensors that detect one-time interactions, such as accelerating or flipping, but neither require active electronics nor power at the time of the interaction. They memorize a pre-defined interaction via an embedded structure filled with a conductive medium (e.g., a liquid). Whether a sensor was exposed to the interaction can be read-out via a capacitive touchscreen. Sensors are printed in a single pass on a consumer-level 3D printer. Through a series of experiments, we show the feasibility of off-line sensing.