Current virtual reality (VR) systems limit users to a single virtual 3D environment (VE) at a time, which restricts their ability to engage with multiple VEs simultaneously. We present SLIVR, a VR-Hub designed to enhance multitasking capabilities by arranging multiple VEs in a regular polygon formation around the user. Each VE is visually represented by life-sized 2D previews that facilitate hub-based interactions. Direct Selection enables users to select objects within a VE directly from the preview, allowing them to remain within the hub. Additionally, Focus-mode enlarges a chosen 2D preview, providing access to objects beyond the initial viewport. In a user study ($\mathrm{N}=18$), we evaluated SLIVR against a conventional grid-based VR app launcher and a hub that utilized SLIVR's visualization but only supported Teleportation interactions. Our findings revealed that SLIVR improved target selection performance by 16.8% overall and by 30.5% when engaging with six VEs, compared to the grid-based app launcher. Notably, hub-based interactions accounted for 94.9% of selections, with Focus-mode being utilized six times more frequently than Teleportation.

This demo explores prioritization techniques to arrange see-through masks in virtual reality (VR). The oval masks show live previews of different virtual environments (VEs) and allow for seamless teleportation into a corresponding VE by putting the mask on the face. Each environment includes a mini-game (e.g., basketball and archery) in which the user has to perform a small task. The arrangement of the masks changes depending on a calculated rating, which considers the time since the game was last played and the game score. We envision this system to help users to multitask in VR. For example, to control multiple characters in VR games, to experience multi-strand (nonlinear) narratives, and to supervise semi-autonomous agents in different VEs.

Movement guidance in virtual reality has many applications ranging from physical therapy, assistive systems to sport learning. These movements range from simple single-limb to complex multi-limb movements. While VR supports many perspectives – e.g., first person and third person – it remains unclear how accurate these perspectives communicate different movements. In a user study (N=18), we investigated the influence of perspective, feedback, and movement properties on the accuracy of movement guidance. Participants had on average an angle error of 6.2° for single arm movements, 7.4° for synchronous two arm movements, and 10.3° for synchronous two arm and leg movements. Furthermore, the results show that the two variants of third-person perspectives outperform a first-person perspective for movement guidance (19.9% and 24.3% reduction in angle errors). Qualitative feedback confirms the quantitative data and shows users have a clear preference for third-person perspectives. Through our findings we provide guidance for designers and developers of future VR movement guidance systems.

Computer-supported posture guidance is used in sports, dance training, expression of art with movements, and learning gestures for interaction. At present, the influence of display types and visualizations have not been investigated in the literature. These factors are important as they directly impact perception and cognitive load, and hence influence the performance of participants. In this paper, we conducted a controlled experiment with 20 participants to compare the use of five display types with different screen sizes: smartphones, tablets, desktop monitors, TVs, and large displays. On each device, we compared three common visualizations for posture guidance: skeletons, silhouettes, and 3d body models. To conduct our assessment, we developed a mobile and cross-platform system that only requires a single camera. Our results show that compared to a smartphone display, larger displays show a lower error (12%). Regarding the choice of visualization, participants rated 3D body models as significantly more usable in comparison to a skeleton visualization.

We envision the human body as a platform for fast take-off and landing of drones in entertainment and professional uses such as medical emergencies, rescue missions, or supporting police units. This new interaction modality challenges our knowledge of human-drone experiences, in which interaction usually occurs at a distance from the body. This work explores important factors for understanding the interplay between drones and humans. We first investigated the suitability of various body locations for landing in an online study (N = 159). Our results, presented as body maps, show that the hand and upper back are particularly well-suited body locations. We further tested these findings in a follow-up study (N = 12), in which participants experienced drones landing on their bodies through carefully designed and pre-recorded 360°videos. This immersion into the landing scenarios helped us to identify common themes and research approaches for different body parts. Taken together, the findings provide first insights into location preferences and themes for drones landing on the human body.

We contribute a novel user- and activity-independent kinematics-based regressive model for continuously predicting ballistic hand movements in virtual reality (VR). Compared to prior work on end-point prediction, continuous hand trajectory prediction in VR enables an early estimation of future events such as collisions between the user’s hand and virtual objects such as UI widgets. We developed and validated our prediction model through a user study with 20 participants. The study collected hand motion data with a 3D pointing task and a gaming task with three popular VR games. Results show that our model can achieve a low Root Mean Square Error (RMSE) of 0.80 cm, 0.85 cm and 3.15 cm from future hand positions ahead of 100 ms, 200 ms and 300 ms respectively across all the users and activities. In pointing tasks, our predictive model achieves an average angular error of 4.0° and 1.5° from the true landing position when 50% and 70% of the way through the movement. A follow-up study showed that the model can be applied to new users and new activities without further training.

Human-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify an agenda for future research. We present a set of challenges for HInt research, formulated over the course of a five-day workshop consisting of 29 experts who have designed, deployed and studied HInt systems. This agenda aims to guide researchers in a structured way towards a more coordinated and conscientious future of human-computer integration.

Accurate sketching in virtual 3D environments is challenging due to aspects like limited depth perception or the absence of physical support. To address this issue, we propose VRSketchPen – a pen that uses two haptic modalities to support virtual sketching without constraining user actions: (1) pneumatic force feedback to simulate the contact pressure of the pen against virtual surfaces and (2) vibrotactile feedback to mimic textures while moving the pen over virtual surfaces. To evaluate VRSketchPen, we conducted a lab experiment with 20 participants to compare (1) pneumatic, (2) vibrotactile and (3) a combination of both with (4) snapping and no assistance for flat and curved surfaces in a 3D virtual environment. Our findings show that usage of pneumatic, vibrotactile and their combination significantly improves 2D shape accuracy and leads to diminished depth errors for flat and curved surfaces. Qualitative results indicate that users find the addition of unconstraining haptic feedback to significantly improve convenience, confidence and user experience.

The body provides many recognizable landmarks due to the underlying skeletal structure and variations in skin texture, elasticity, and color. The visual and spatial cues of such body landmarks can help in localizing on-body interfaces, guide input on the body, and allow for easy recall of mappings. Our main contribution are SkinMarks, novel skin-worn I/O devices for precisely localized input and output on fine body landmarks. SkinMarks comprise skin electronics on temporary rub-on tattoos. They conform to fine wrinkles and are compatible with strongly curved and elastic body locations. We identify five types of body landmarks and demonstrate novel interaction techniques that leverage SkinMarks' unique touch, squeeze and bend sensing with integrated visual output. Finally, we detail on the conformality and evaluate sub-millimeter electrodes for touch sensing. Taken together, SkinMarks expands the on-body interaction space to more detailed, highly curved and challenging areas on the body.

We propose iSkin, a novel class of skin-worn sensors for touch input on the body. iSkin is a very thin sensor overlay, made of biocompatible materials, and is flexible and stretchable. It can be produced in different shapes and sizes to suit various locations of the body such as the finger, forearm, or ear. Integrating capacitive and resistive touch sensing, the sensor is capable of detecting touch input with two levels of pressure, even when stretched by 30% or when bent with a radius of 0.5cm. Furthermore, iSkin supports single or multiple touch areas of custom shape and arrangement, as well as more complex widgets, such as sliders and click wheels. Recognizing the social importance of skin, we show visual design patterns to customize functional touch sensors and allow for a visually aesthetic appearance. Taken together, these contributions enable new types of on-body devices. This includes finger-worn devices, extensions to conventional wearable devices, and touch input stickers, all fostering direct, quick, and discreet input for mobile computing.

This paper contributes results from an empirical study of on-skin input, an emerging technique for controlling mobile devices. Skin is fundamentally different from off-body touch surfaces, opening up a new and largely unexplored interaction space. We investigate characteristics of the various skin-specific input modalities, analyze what kinds of gestures are performed on skin, and study what are preferred input locations. Our main findings show that (1) users intuitively leverage the properties of skin for a wide range of more expressive commands than on conventional touch surfaces; (2) established multi-touch gestures can be transferred to on-skin input; (3) physically uncomfortable modalities are deliberately used for irreversible commands and expressing negative emotions; and (4) the forearm and the hand are the most preferred locations on the upper limb for on-skin input. We detail on users' mental models and contribute a first consolidated set of on-skin gestures. Our findings provide guidance for developers of future sensors as well as for designers of future applications of on-skin input.

Researchers have developed interaction concepts based on mobile projectors. Yet pursuing work in this area - particularly in building projector-based interactions techniques within an application - is cumbersome and time-consuming. To mitigate this problem, we contribute ProjectorKit, a flexible open-source toolkit that eases rapid prototyping mobile projector interaction techniques.