Gait is a distinctive behavioral trait, yet its vulnerability against imitation remains underexplored in immersive environments. We present a study investigating how real-time visual feedback in virtual reality (VR) influences a person’s ability to mimic another’s gait. Through two experiments, we first identify the most usable feedback design (N = 8), then evaluate its impact on imitation performance compared to a baseline without feedback (N = 18). We analyze positional and rotational similarity between participants and target avatars, examining the influence of avatar–user gender matching and repeated practice. Our findings reveal that visual feedback significantly improves rotational alignment and that practice leads to measurable improvements in mimicry accuracy. We discuss implications for avatar embodiment, personalization in VR applications, and potential considerations for behavioral biometric systems. We also contribute a publicly available dataset of gait mimicry in VR, supporting further research on motion learning and imitation.

Spatial Augmented Reality (SAR) can enhance learning experiences through interactive, real-time digital information overlays. Using SAR, content can be projected directly onto physical paper to provide students with in situ task feedback. Our work explores how students perceive different types of SAR feedback. We first identified feedback methods and dimensions from a literature review. We then conducted an expert focus group (N = 5) of professionals who had backgrounds in education and teaching experience. With the focus group, we aimed to expand on the literature review results to identify feedback modalities and dimensions commonly used in classrooms today. Next, we performed a field study (N = 16) with high school students in which we compared the perception of different feedback modalities (text, image, video) and familiarity (neutral, unfamiliar, familiar) in a classroom setting. Our results revealed that perceived user distraction and novelty are significantly affected by feedback modality through a large effect, with videos being perceived as more distracting and more novel. Familiar, trusted individuals best deliver positive feedback, whereas negative feedback from people should be avoided. We discuss the usage of feedback modalities in various contexts, providing a foundation for future use of SAR feedback in education.

Behavioral Biometrics in Virtual Reality (VR) allow for implicit user identification, as the head- and hand-movements that can be captured from the head-mounted display and the controllers are highly descriptive of the user’s true identity. Such body movements have been explored in the past; however, to date, it is unclear how they perform in settings where more than one person interacts in a shared virtual environment. In this work, we explored through a user study (N=40) how behavioral biometrics in VR change when one or more persons interact with each other in a shared virtual environment and whether this is influenced by the nature of the interaction itself. We find that user identification is possible with up to 83.38 % by applying deep learning models, and that particularly cooperative interactions between multiple VR users lead to highly identifiable body movements. Our results help in advancing behavioral biometrics for seamless user identification in VR, as a viable alternative to using PINs and passwords.

Behavioral Biometrics in Virtual Reality (VR) enable implicit user identification by leveraging the motion data of users’ heads and hands from their interactions in VR. This spatiotemporal data forms a Kinetic Signature, which is a user-dependent behavioral biometric trait. Although kinetic signatures have been widely used in recent research, the factors contributing to their degree of identifiability remain mostly unexplored. Drawing from existing literature, this work systematically examines the influence of static and dynamic components in human motion. We conducted a user study (N = 24) with two sessions to reidentify users across different VR sports and exercises after one week. We found that the identifiability of a kinetic signature depends on its inherent static and dynamic factors, with the best combination allowing for 90.91% identification accuracy after one week had passed. Therefore, this work lays a foundation for designing and refining movement-based identification protocols in immersive environments.

There is an endeavor in the Human-Computer Interaction (HCI) community to create novel authentication schemes so that passwords finally become obsolete and practical security is enhanced. For this purpose, researchers combine behavioral biometrics with deep learning. However, because the process of creating neural networks is inherently complex and each model architecture has certain limitations, implementing research prototypes is a time-consuming and challenging task. Therefore, we present useckit, an open-source toolkit that provides deep learning algorithms to support the creation of scientific evaluations in authentication research. Useckit provides multiple paradigms for implementing user verification and identification, functions to calculate common metrics, and neural network architectures founded in literature. It is written in Python and supports researchers and practitioners in creating, implementing, and rigorously evaluating novel deep learning-based authentication schemes.

Nowadays, Augmented and Virtual Reality devices are widely available and are often shared among users due to their high cost. Thus, distinguishing users to offer personalized experiences is essential. However, currently used explicit user authentication (e.g., entering a password) is tedious and vulnerable to attack. Therefore, this work investigates the feasibility of implicitly identifying users by their hand tracking data. In particular, we identify users by their uni- and bimanual finger behavior gathered from their interaction with eight different universal interface elements, such as buttons and sliders. In two sessions, we recorded the tracking data of 16 participants while they interacted with various interface elements in Augmented and Virtual Reality. We found that user identification is possible with up to 95% accuracy across sessions using an explainable machine learning approach. We conclude our work by discussing differences between interface elements, and feature importance to provide implications for behavioral biometric systems.A video abstract of this work is available online at: https://identifying-users-by-hand-tracking-data.hcigroup.de

Scanning everyday objects with depth sensors is the state-of-the-art approach to generating point clouds for realistic 3D representations. However, the resulting point cloud data suffers from outliers and contains irrelevant data from neighboring objects. To obtain only the desired 3D representation, additional manual segmentation steps are required. In this paper, we compare three different technology classes as independent variables (desktop vs. tablet vs. virtual reality) in a within-subject user study (N = 18) to understand their effectiveness and efficiency for such segmentation tasks. We found that desktop and tablet still outperform virtual reality regarding task completion times, while we could not find a significant difference between them in the effectiveness of the segmentation. In the post hoc interviews, participants preferred the desktop due to its familiarity and temporal efficiency and virtual reality due to its given three-dimensional representation.

Robots play a vital role in modern automation, with applications in manufacturing and healthcare. Collaborative robots integrate human and robot movements. Therefore, it is essential to ensure that interactions involve qualified, and thus identified, individuals. This study delves into a new approach: identifying individuals through robot arm movements. Different from previous methods, users guide the robot, and the robot senses the movements via joint sensors. We asked 18 participants to perform six gestures, revealing the potential use as unique behavioral traits or biometrics, achieving F1-score up to 0.87, which suggests direct robot interactions as a promising avenue for implicit and explicit user identification.

Behavioral biometrics has recently become a viable alternative method for user identification in Virtual Reality (VR). Its ability to identify users based solely on their implicit interaction allows for high usability and removes the burden commonly associated with security mechanisms. However, little is known about the temporal stability of behavior (i.e., how behavior changes over time), as most previous works were evaluated in highly controlled lab environments over short periods. In this work, we present findings obtained from a remote field study (N = 15) that elicited data over a period of eight weeks from a popular VR game. We found that there are changes in people’s behavior over time, but that two-session identification still is possible with a mean F1-score of up to 71%, while an initial training yields 86%. However, we also see that performance can drop by up to over 50 percentage points when testing with later sessions, compared to the first session, particularly for smaller groups. Thus, our findings indicate that the use of behavioral biometrics in VR is convenient for the user and practical with regard to changing behavior and also reliable regarding behavioral variation.

The global COVID-19 pandemic created a constant need for hand disinfection. While it is still essential, disinfection use is declining with the decrease in perceived personal risk (e.g., as a result of vaccination). Thus this work explores using different visual cues to act as reminders for hand disinfection. We investigated different public display designs using (1) paper-based only, adding (2) screen-based, or (3) projection-based visual cues. To gain insights into these designs, we conducted semi-structured interviews with passersby (N=30). Our results show that the screen- and projection-based conditions were perceived as more engaging. Furthermore, we conclude that the disinfection process consists of four steps that can be supported: drawing attention to the disinfection station, supporting the (subconscious) understanding of the interaction, motivating hand disinfection, and performing the action itself. We conclude with design implications for technology-supported disinfection.

In this work, we explore attacker behavior during shoulder surfing. As such behavior is often opportunistic and difficult to observe in real world settings, we leverage the capabilities of virtual reality (VR). We recruited 24 participants and observed their behavior in two virtual waiting scenarios: at a bus stop and in an open office space. In both scenarios, participants shoulder surfed private screens displaying different types of content. From the results we derive an understanding of factors influencing shoulder surfing behavior, reveal common attack patterns, and sketch a behavioral shoulder surfing model. Our work suggests directions for future research on shoulder surfing and can serve as a basis for creating novel approaches to mitigate shoulder surfing.

This paper describes the experimental framework and results of the IJCB 2022 Mobile Behavioral Biometrics Competition (MobileB2C). The aim of MobileB2C is bench-marking mobile user authentication systems based on behavioral biometric traits transparently acquired by mobile devices during ordinary Human-Computer Interaction (HCI), using a novel public database, BehavePassDB11https://github.com/BiDAlab/MobileB2C_BehavePassDE, and a standard experimental protocol. The competition is divided into four tasks corresponding to typical user activities: keystroke, text reading, gallery swiping, and tapping. The data are composed of touchscreen data and several background sensor data simultaneously acquired. “Random” (different users with different devices) and “skilled” (different user on the same device attempting to imitate the legitimate one) impostor scenarios are considered. The results achieved by the participants show the feasibility of user authentication through behavioral biometrics, although this proves to be a non-trivial challenge. MobileB2C will be established as an on-going competition22https://sites.google.com/view/mobileb2c/.

Nowadays, deep learning models enable numerous safety-critical applications, such as biometric authentication, medical diagnosis support, and self-driving cars. However, previous studies have frequently demonstrated that these models are attackable through slight modifications of their inputs, so-called adversarial attacks. Hence, researchers proposed investigating examples of these attacks with explainable artificial intelligence to understand them better. In this line, we developed an expert tool to explore adversarial attacks and defenses against them. To demonstrate the capabilities of our visualization tool, we worked with the publicly available CIFAR-10 dataset and generated one-pixel attacks. After that, we conducted an online evaluation with 16 experts. We found that our tool is usable and practical, providing evidence that it can support understanding, explaining, and preventing adversarial examples.

Modern ubiquitous computing environments are increasingly populated with smart devices that need to know the identity of users interacting with them. At the same time, the number of authentications that a user needs to perform increases, as nowadays devices such as smart TVs require authentication which was not the case in earlier times. Even for single-person households, the need to authenticate against present smart devices in the environment appears at regular intervals, ranging from TVs to voice assistants, to gaming consoles. To reduce the need for repeated authentication, we explore the concept of a system that allows the sharing of users’ authenticated identity information between smart devices, similar to the concept of Single-Sign-On on the internet. Following a preliminary field study, we show that such a system can decrease the number of necessary authentications in a ubiquitous computing environment by 84.4%, increasing usability and security.

Augmented Reality (AR) technology enables users to superpose virtual content onto their environments. However, interacting with virtual content while mobile often requires users to perform interactions in mid-air, resulting in a lack of haptic feedback. Hence, in this work, we present the ARm Haptics system, which is worn on the user's forearm and provides 3D-printed input modules, each representing well-known interaction components such as buttons, sliders, and rotary knobs. These modules can be changed quickly, thus allowing users to adapt them to their current use case. After an iterative development of our system, which involved a focus group with HCI researchers, we conducted a user study to compare the ARm Haptics system to hand-tracking-based interaction in mid-air (baseline). Our findings show that using our system results in significantly lower error rates for slider and rotary input. Moreover, use of the ARm Haptics system results in significantly higher pragmatic quality and lower effort, frustration, and physical demand. Following our findings, we discuss opportunities for haptics worn on the forearm.

Virtual Reality (VR) is becoming increasingly popular both in the entertainment and professional domains. Behavioral biometrics have recently been investigated as a means to continuously and implicitly identify users in VR. Applications in VR can specifically benefit from this, for example, to adapt virtual environments and user interfaces as well as to authenticate users. In this work, we conduct a lab study (N = 16) to explore how accurately users can be identified during two task-driven scenarios based on their spatial movement. We show that an identification accuracy of up to 90% is possible across sessions recorded on different days. Moreover, we investigate the role of users’ physiology in behavioral biometrics by virtually altering and normalizing their body proportions. We find that body normalization in general increases the identification rate, in some cases by up to 38%; hence, it improves the performance of identification systems.

Shoulder surfing is an omnipresent risk for smartphone users. However, investigating these attacks in the wild is difficult because of either privacy concerns, lack of consent, or the fact that asking for consent would influence people’s behavior (e.g., they could try to avoid looking at smartphones). Thus, we propose utilizing 360-degree videos in Virtual Reality (VR), recorded in staged real-life situations on public transport. Despite differences between perceiving videos in VR and experiencing real-world situations, we believe this approach to allow novel insights on observers’ tendency to shoulder surf another person’s phone authentication and interaction to be gained. By conducting a study (N=16), we demonstrate that a better understanding of shoulder surfers’ behavior can be obtained by analyzing gaze data during video watching and comparing it to post-hoc interview responses. On average, participants looked at the phone for about 11% of the time it was visible and could remember half of the applications used.

Identifying users of a Virtual Reality (VR) headset provides designers of VR content with the opportunity to adapt the user interface, set user-specific preferences, or adjust the level of difficulty either for games or training applications. While most identification methods currently rely on explicit input, implicit user identification is less disruptive and does not impact the immersion of the users. In this work, we introduce a biometric identification system that employs the user’s gaze behavior as a unique, individual characteristic. In particular, we focus on the user’s gaze behavior and head orientation while following a moving stimulus. We verify our approach in a user study. A hybrid post-hoc analysis results in an identification accuracy of up to 75 % for an explainable machine learning algorithm and up to 100 % for a deep learning approach. We conclude with discussing application scenarios in which our approach can be used to implicitly identify users.

Winter sports like skiing are becoming increasingly popular for both competitive and recreational activities. To minimize the risk of injury, new innovations in skiing equipment have been developed in recent years. However, unexpected slope conditions can still increase risks during skiing. The static categorisation of ski slopes in winter sports resorts does not take into account dynamic changes of difficulty due to high traffic volumes or sudden weather changes. Up to now, efforts have been made to measure the current conditions via satellite imaging or installations on the slope. However, this requires intervention in nature and causes high maintenance costs. To solve these issues we present our preliminary design of a wearable system to let skiers implicitly measure current slope conditions during their skiing experience. Audio and motion data are recorded from a prototype mounted on a ski boot. We show that the data generated by the prototype can be successfully classified with a neural network. We collected data from a skiing activity to demonstrate our concept and discuss the identified challenges in fitting the proposed approach to winter sports equipment.