In virtual reality, users are typically represented by and interact through a virtual body. Research frequently manipulates different features of these bodies. We analyze 208 studies to identify what aspects of virtual bodies are manipulated, how these manipulations affect interaction and other outcomes, and why they are assumed to do so. Based on the analysis, we propose a design space comprising seven types of visual manipulations: appearance, size, morphology, viewpoint, transfer, remapping, and control. We also synthesize findings on their effects—ranging from task performance to physiological responses and social outcomes—and examine the theories used to explain them, such as embodiment, Proteus effect, and presence. The design space helps researchers identify key variables and their interconnections in design and empirical research of virtual bodies. The synthesis further reveals unexplored causal connections and highlights theories that may account for observed effects.

VR interaction techniques define mappings between physical movements and virtual outcomes. While some mappings are learned through the adaptation of existing movement strategies, others require acquiring entirely new control policies. Drawing on motor learning theory, we introduce a design space that organizes these mappings into three families and provides a basis for reasoning about how the mappings are learned. To examine learning within individual families and their compositions, we start with two simple hand-based mappings, mirror reversal and cross-hand control, and their combination, allowing us to probe the design space in a controlled experiment with 96 participants. The mappings differ in initial difficulty, but participants achieve comparable overall learning. In the combined condition, prior exposure produces mapping specific start-up advantages. We discuss how this design space can support the analysis and design of VR mapping techniques.

A VR body can move faster than its user, making actions like reaching more efficient. We propose a VR body that not only moves faster during reaching, but also starts moving before the user has decided which target to reach for. However, it is unclear whether such a VR body would speed up choices, since moving towards a wrong target might cause confusion, or influence users’ choices. To explore these questions, we built a faster VR body prototype, focusing on an accelerated hand and arm, in a choice task. Thirty-four participants viewed random-dot displays to judge the overall motion direction and indicated their choice by pressing the corresponding button. Task difficulty was varied to influence choice uncertainty. Results showed that choice time decreased when the virtual body was 0.1 seconds ahead of the physical body, but increased when it was 0.3 seconds ahead. Users’ choice distributions showed no significant differences.

Tendon vibration can create movement illusions: vibrating the biceps tendon induces an illusion of extending the arm, while vibrating the triceps tendon induces an illusion of flexing the arm. However, it is unclear how to create and integrate such illusions shown in neuroscience to interaction techniques in virtual reality (VR). We first design a motor setup for tendon vibration. Study 1 validates that the setup induces movement illusions which on average create a 5.26 cm offset in active arm movements. Study 2 shows that tendon vibration improves the detection thresholds of visual motion gains often used in VR interaction techniques by 0.22. A model we developed in Study 2 predicts the effects of tendon vibration and is used in a biomechanical simulation to demonstrate the detection thresholds across typical reaching tasks in VR.

Presence appears an important concept for virtual reality (VR): It is frequently measured with questionnaires, and theory and methods about it have been discussed in numerous works. Yet, it is unclear how to actually work with this concept: Why is presence important to measure, how to choose an appropriate questionnaire, and what to conclude about it based on findings? To answer these questions, we review how the concept is put to work in 288 VR papers from 2023 measuring presence with questionnaires. Our findings include that measuring presence is often motivated by another construct, such as user experience; the reasons for choosing a specific questionnaire are often weak or not reported at all; and high presence values are frequently used simply to validate an interaction technique. We propose recommendations for working with presence and formulate questions to direct future research.

Designing efficient selection techniques for graphical user interfaces (GUIs) is fundamental in HCI research.We derive selection techniques based on the multiple process model, a theory that details the motor control processes during goal-directed movements.Specifically, we deduce three theoretical assumptions on how control processes of pre-planning, impulse control, and limb-target control could influence selection movements when adjusting GUI elements, including visual feedback, cursor position, and target position.Corresponding to our assumptions, we develop three techniques that hide the cursor when a target is highlighted, snap the cursor when selection begins, and expand clustered objects during selection movements.After that, we pre-register the assumptions and research methodology and evaluate the techniques in three crowdsourcing-based pointing studies.Our results show that all techniques improved the selection efficiency compared to established baselines.We further discuss the design implications and reflect on how we derived techniques from theory.

Perceived Continuity CRE f) FE STM CRE FE STM a) 1 2 3 3 4 c) b) d) Figure 1: a) Different sensitivity areas across the hand overlaid with pacinian receptors (dots).We create tactile shapes at low (c) and high (b) sensitivity areas, as well as across areas (d and e), comparing their perceived intensity and continuity when using spatiotemporal modulation (STM) and tactile illusions (funneling effect (FE) and cutaneous rabbit effect (CRE) ) (f).

The user interface is a central concept in human–computer interaction, but peculiarly fuzzy. This affects discussions of the fundamentals of our discipline and the positioning of our work. We propose a model of the user interface that consists of four elements: Devices, Interaction Techniques, Representations, and Assemblies (DIRA). We explain their roles in the user interface and discuss some associated concerns about evaluation and design. We then show how to use the model to describe the elements of user interfaces (with examples that include a menu, a fisheye interface, and notifications) and to analyze the central characteristics of user interface paradigms (including tangible user interfaces and mixed reality). Finally, we discuss how describing user interfaces with the model can drive their design and evaluation. • The concept of user interface (UI) is central to HCI. • The elements of UI are Devices, Interaction Techniques, Representations, and Assemblies (DIRA). • The DIRA model can help describing and analyzing UIs and drive their evaluation and design.

Haptic experiences – technology-mediated experiences of touch – are complex and enigmatic; they are intertwined with the user’s motivations, context, and past experiences and are deeply interwoven with other sensory modalities. To better understand such experiences, we develop the Inference-Design Model for Haptic Experience. The model separates stimulus, sensation, and experience and identifies two diametrical processes: inference and design. Inference is the cognitive process through which humans interpret a sensory environment, while design involves the creation of such a sensory environment. Grounded in philosophical, neuroscientific, and psychological literature, we discuss the formation of haptic experiences facilitated through the structure and components of the Inference-Design Model. We use the model to present six key observations about the inference of haptic experiences and the design processes that enable them. The observations and their implications for researchers contribute to a fundamental understanding of haptic experiences in a model-centric research paradigm.

The “doorway effect” predicts that crossing an environmental boundary affects memory negatively. In virtual reality (VR), we can design the crossing and the appearance of such boundaries in non-realistic ways. However, it is unclear whether locomotion techniques like teleportation, which avoid crossing the boundary altogether, still induce the effect. Furthermore, it is unclear how different appearances of a doorway act as a boundary and thus induce the effect. To address these questions, we conducted two lab studies. First, we conceptually replicated prior doorway effect studies in VR using natural walking and teleportation. Second, we investigated the effect of five doorway visualizations, ranging from doors to portals. The results show no difference in object recognition performance due to the presence of a doorway, locomotion technique, or doorway visualization. We discuss the implications of these findings on the role of boundaries in event-based memory and the design of boundary interactions in VR.

Walking-based locomotion techniques in virtual reality (VR) can use redirection to enable walking in a virtual environment larger than the physical one. This results in a mismatch between the perceived virtual and physical movement, which is known to cause VR sickness. However, it is unclear if different types of walking techniques (e.g., resetting, reorientation, or self-overlapping spaces) affect VR sickness differently. To address this, we conducted a systematic review and meta-analysis of 96 papers published in 2016–2022 that measure VR sickness in walking-based locomotion. We find different VR sickness effects between types of redirection and between normal walking and redirection. However, we also identified several problems with the use and reporting of VR sickness measures. We discuss the challenges in understanding VR sickness differences between walking techniques and present guidelines for measuring VR sickness in locomotion studies.

Virtual reality (VR) techniques can modify how physical body movements are mapped to the virtual body. However, it is unclear how users learn such mappings and, therefore, how the learning process may impede interaction. To understand and quantify the learning of the techniques, we design new metrics explicitly for VR interactions based on the motor learning literature. We evaluate the metrics in three object selection and manipulation tasks, employing linear-translational and nonlinear-rotational gains and finger-to-arm mapping. The study shows that the metrics demonstrate known characteristics of motor learning similar to task completion time, typically with faster initial learning followed by more gradual improvements over time. More importantly, the metrics capture learning behaviors that task completion time does not. We discuss how the metrics can provide new insights into how users adapt to movement mappings and how they can help analyze and improve such techniques.

Object selection has been explored extensively in the VR research literature. However, the research is typically conducted in constrained experimental setups. It remains unclear whether the designed selection techniques fit the prevalent practical uses and whether the experimental tasks represent important challenges in real applications. To identify and help bridge these gaps, we surveyed current consumer VR applications, containing 206 popular VR game and 3D modeling applications. We extracted 1300+ selection scenarios based on video analyses of these applications and derived a taxonomy to understand common patterns on where and how selections occur. Our findings reveal significant gaps in selection tasks and techniques between research and consumer applications. We also present an interactive visualization tool to help researchers explore the VR object selection scenarios. Finally, we discuss how our work can help researchers and developers evaluate techniques in meaningful tasks and drive the design of techniques.

Most contemporary Virtual Reality (VR) experiences are made for standing users. However, when a user is lying down—either by choice or necessity—it is unclear how they can walk around, dodge obstacles, or grab distant objects. We rotate the virtual coordinate space to study the movement requirements and user experience of using VR while lying down. Fourteen experienced VR users engaged with various popular VR applications for 40 minutes in a study using a think-aloud protocol and semi-structured interviews. Thematic analysis of captured videos and interviews reveals that using VR while lying down is comfortable and usable and that the virtual perspective produces a potent illusion of standing up. However, commonplace movements in VR are surprisingly difficult when lying down, and using alternative interactions is fatiguing and hampers performance. To conclude, we discuss design opportunities to tackle the most significant challenges and to create new experiences.

We introduce OVRlap, a VR interaction technique that lets the user perceive multiple places at the same time from a first-person perspective. OVRlap achieves this by overlapping viewpoints. At any time, only one viewpoint is active, meaning that the user may interact with objects therein. Objects seen from the active viewpoint are opaque, whereas objects seen from passive viewpoints are transparent. This allows users to perceive multiple locations at once and easily switch to the one in which they want to interact. We compare OVRlap and a single-viewpoint technique in a study where 20 participants complete object-collection and monitoring tasks. We find that in both tasks, participants are significantly faster and move their head significantly less with OVRlap. We propose how the technique might be improved through automated switching of the active viewpoint and intelligent viewpoint rendering.

Mid-air haptic stimulation can enrich user experience during human-computer interaction. However, the design space of such stimuli is large due to the number and range of stimulation parameters. It therefore remains difficult for designers to select a stimulus to induce an intended experience. We derive a mapping for mid-air experiences based on two user studies. In the first study, participants rated 36 stimuli varied across three parameters (frequency, pattern, and repetitions). These ratings allowed us to determine a set of five experientially distinct stimuli. In the second study, participants vocalized their experiences with those five stimuli. This allows us to generate a mapping of 17 sensations and 23 experiences related to the stimuli. Finally, we discuss how the mapping can inform designers and researchers working with mid-air haptic technologies.

Sense of control is increasingly used as a measure of quality in human-computer interaction. Control has been investigated mainly at a high level, using subjective questionnaire data, but also at a low level, using objective data on participants’ sense of agency. However, it remains unclear how differences in higher level, experienced control reflect lower level sense of control. We study that link in two experiments. In the first one we measure the low-level sense of agency with button, touchpad, and on-skin input. The results show a higher sense of agency with on-skin input. In the second experiment, participants played a simple game controlled with the same three inputs. We find that on-skin input results in both increased sense and experience of control compared to touchpad input. However, the corresponding difference is not found between on-skin and button input, whereas the button performed better in the experiment task. These results suggest that other factors of user experience spill over to the experienced control at rates that overcome differences in the sense of control. We discuss the implications for using subjective measures about the sense of control in evaluating qualities of interaction.

The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefits over the state-of-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate findings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them. We also identify research directions for improving evaluation methods and offer ideas for how to make studies more ecologically valid and rigorous.

Selection and manipulation in virtual reality often happen using an avatar’s hands. However, objects outside the immediate reach require effort to select. We develop a target selection technique called Ninja Hands. It maps the movement of a single real hand to many virtual hands, decreasing the distance to targets. We evaluate Ninja Hands in two studies. The first study shows that compared to a single hand, 4 and 8 hands are significantly faster for selecting targets. The second study complements this finding by using a larger target layout with many distractors. We find no decrease in selection time across 8, 27, and 64 hands, but an increase in the time spent deciding which hand to use. Thereby, net movement time still decreases significantly. In both studies, the physical motion exerted also decreases significantly with more hands. We discuss how these findings can inform future implementations of the Ninja Hands technique.

Virtual reality (VR) allows users to interact similarly to how they do in the physical world, such as touching, moving, and pointing at objects. To select objects at a distance, most VR techniques rely on casting a ray through one or two points located on the user’s body (e.g., on the head and a finger), and placing a cursor on that ray. However, previous studies show that such rays do not help users achieve optimal pointing accuracy nor correspond to how they would naturally point. We seek to find features, which would best describe natural pointing at distant targets. We collect motion data from seven locations on the hand, arm, and body, while participants point at 27 targets across a virtual room. We evaluate the features of pointing and analyse sets of those for predicting pointing targets. Our analysis shows an 87% classification accuracy between the 27 targets for the best feature set and a mean distance of 23.56 cm in predicting pointing targets across the room. The feature sets can inform the design of more natural and effective VR pointing techniques for distant object selection.

Virtual Reality allows users to embody avatars that do not match their real bodies. Earlier work has selected changes to the avatar arbitrarily and it therefore remains unclear how to change avatars to improve users' performance. We propose a systematic approach for iteratively adapting the avatar to perform better for a given task based on users' performance. The approach is evaluated in a target selection task, where the forearms of the avatar are scaled to improve performance. A comparison between the optimised and real arm lengths shows a significant reduction in average tapping time by 18.7%, for forearms multiplied in length by 5.6. Additionally, with the adapted avatar, participants moved their real body and arms significantly less, and subjective measures show reduced physical demand and frustration. In a second study, we modify finger lengths for a linear tapping task to achieve a better performing avatar, which demonstrates the generalisability of the approach.

Tool use extends people's representations of the immediately actionable space around them. Physical tools thereby become integrated in people's body schemas. We introduce a measure for tool extension in HCI by using a visual-tactile interference paradigm. In this paradigm, an index of tool extension is given by response time differences between crossmodally congruent and incongruent stimuli; tactile on the hand and visual on the tool. We use this measure to examine if and how findings on tool extension apply to interaction with computer-based tools. Our first experiment shows that touchpad and mouse both provide tool extension over a baseline condition without a tool. A second experiment shows a higher degree of tool extension for a realistic avatar hand compared to an abstract pointer for interaction in virtual reality. In sum, our measure can detect tool extension with computer-based tools and differentiate interfaces by their degree of extension.

Touch accuracy is not just dependent on the performance of the touch sensor itself. Instead, aspects like phone grip or occlusion of the screen have been shown to also have an influence on accuracy. Yet, these are all dependent on one underlying factor: the size and proportions of the user's hand. To better understand touch input, we investigate how 11 hand features influence accuracy. We find that thumb length in particular correlates significantly with touch accuracy and accounts for about 12% of touch error variance. Furthermore, we show that measures of some higher level interactions also correlate with hand size.

Previous work in VR has demonstrated how individual physical objects can represent multiple virtual objects in different locations by redirecting the user's hand. We show how individual objects can represent multiple virtual objects of different sizes by resizing the user's grasp. We redirect the positions of the user's fingers by visual translation gains, inducing an illusion that can make physical objects seem larger or smaller. We present a discrimination experiment to estimate the thresholds of resizing virtual objects from physical objects, without the user reliably noticing a difference. The results show that the size difference is easily detected when a physical object is used to represent an object less than 90% of its size. When physical objects represent larger virtual objects, however, then scaling is tightly coupled to the physical object's size: smaller physical objects allow more virtual resizing (up to a 50% larger virtual size). Resized Grasping considerably broadens the scope of using illusions to provide rich haptic experiences in virtual reality.

Input on the skin is emerging as an interaction style. At CHI 2012, Coyle and colleagues identified an increase in the sense of agency (SoA) as one benefit of skin input. However, their study only compared skin input to button presses and has not, to our knowledge, been replicated. Therefore, we had 24 participants compare skin input to both button presses and touchpad input, measuring SoA using the Libet Clock paradigm. We replicate previous findings regarding increased SoA in skin versus button input and also find that SoA for skin is significantly increased compared to touchpad input. Interview data addressing subjective experience further support these findings. We discuss agency and the experiences associated with skin input, as well as differences to input with non-skin devices.

Advances in sensing technologies allow for using the forearm as a touch surface to give input to off-skin displays. However, it is unclear how users perceive the mapping between an on-skin input area and an off-skin display area. We empirically describe such mappings to improve on-skin interaction. We collected discrete and continuous touch data in a study where participants mapped display content from an AR headset, a smartwatch, and a desktop display to their forearm. We model those mappings and estimate input accuracy from the spreads of touch data. Subsequently, we show how to use the models for designing touch surfaces to the forearm for a given display area, input type, and touch resolution.

The human skin provides an ample, always-on surface for input to smart watches, mobile phones, and remote displays. Using touch on bare skin to issue commands, however, requires users to recall the location of items without direct visual feedback. We present an in-depth study in which participants placed 30 items on the hand and forearm and attempted to recall their locations. We found that participants used a variety of landmarks, personal associations, and semantic groupings in placing the items on the skin. Although participants most frequently used anatomical landmarks (e.g., fingers, joints, and nails), recall rates were higher for items placed on personal landmarks, including scars and tattoos. We further found that personal associations between items improved recall, and that participants often grouped important items in similar areas, such as family members on the nails. We conclude by discussing the implications of our findings for design of skin-based interfaces.

We present a predictive model for the functional area of the thumb on a touchscreen surface: the area of the interface reachable by the thumb of the hand that is holding the device. We derive a quadratic formula by analyzing the kinematics of the gripping hand. Model fit is high for the thumb-motion trajectories of 20 participants. The model predicts the functional area for a given 1) surface size, 2) hand size, and 3) position of the index finger on the back of the device. Designers can use this model to ensure that a user interface is suitable for interaction with the thumb. The model can also be used inversely - that is, to infer the grips assumed by a given user interface layout.

Large interfaces are fixed to a certain use context, for example to a physical smart space. Mobile counterparts of public interfaces can allow user to continue interacting with the content also when leaving the space. However, wall applications make use of a large display surface and fitting the same user interface to the constraints of a mobile screen is challenging. Starting with Bubble Wall -- an information exploration application for multitouch walls -- we developed interfaces for browsing the same content graphs on mobile devices. A comparison study in exploration and navigation tasks was conducted with two mobile interfaces reproducing the large screen interactions: BubbleSpace with a more faithful redesign and BubblesDial reducing the interactions for better fit to a small screen. The BubblesDial scored significantly better in usability and performance evaluation, especially when priming with use of the Bubble Wall. We also present implications for the redesign of these large content graph interactions for mobile use.

Everyday activities often involve using an interactive device while one is handling various other physical objects (wallets, bags, doors, pens, mugs, etc.). This paper presents the Manual Multitasking Test, a test with 12 conditions emulating manual demands of everyday multitasking situations. It allows experimenters to expose the effects of design on "manual flexibility": users' ability to reconfigure the sensorimotor control of arms, hands, and fingers in order to regain the high performance levels they experience when using the device on its own. The test was deployed for pointing devices on laptops and Qwerty keyboards of mobile devices. In these studies, we identified facilitative design features whose absence explains, for example, why the mouse and stylus function poorly in multi-object performance. The issue deserves more attention, because interfaces that are nominally similar (e.g., "one-handed input") can vary dramatically in terms of "ease of juggling".

Studies have reported negative effects of walking on mobile human---computer interaction when compared to standing or sitting. However, the quantitative relationship between walking speed and user performance is unknown. In the study described here, we varied walking speed on a treadmill and measured effects on discrete aiming movements on a touchscreen interface. Their relationship was found to be non-linear with a local optimum: when walking at 40--80% of one's preferred walking speed (PWS), target acquisition performance plateaus, indicating optimal trade-off between speed and interaction. Accelerometer data showed that, despite increasing hand oscillation, users were able to maintain stable interaction performance at 74% of PWS. Interestingly, this speed coincides with the speed users spontaneously walk when interacting with a mobile device.

Most interactive tasks engage more than one of the user's exteroceptive senses and are therefore multimodal. In real world situations with multitasking and distractions, the key aspect of multimodality is not which modalities can be allocated to the interactive task but which are free to be allocated to something else. We present the multi¬modal flexibility index (MFI), calculated from changes in users' performance induced by blocking of sensory modalities. A high score indicates that the highest level of performance is achievable regardless of the modalities available and, conversely, a low score that performance will be severely hampered unless all modalities are allocated to the task. Various derivatives describe unimodal and bimodal effects. Results from a case study (mobile text entry) illustrate how an interface that is superior to others in absolute terms is the worst from the multimodal flexibility perspective. We discuss the suitability of MFI for evaluation of interactive prototypes.