In virtual reality (VR) experiences, mismatches between reality and virtuality are usually undesirable, as they can disrupt immersion and induce cybersickness. However, when carefully controlled, they may expand the design space of VR. This research investigates perceptual detection thresholds for mismatches between real and virtual inclinations during cycling in VR. Using a custom simulation, N = 30 participants cycled through a virtual city while physical and visual inclinations were independently manipulated. Real inclinations were implemented with a tilting indoor bike, providing vestibular and proprioceptive feedback, while virtual inclinations within the simulated environment were presented visually. A multiple staircase procedure derived estimates for perceptual thresholds that approximate which mismatches in visual and physical inclination were still perceived as congruent. These thresholds reveal a window of perceived congruence before mismatches become noticeable to users. These findings advance understanding of sensory integration in VR cycling and inform applications in immersive training, exergames, and rehabilitation systems.

In the context of rising sedentary lifestyles, this paper investigates the efficacy of "Situated Artifacts"in promoting physical activity. We designed two artifacts that display users' physical activity data within their homes - one physical and one digital. We conducted a 9-week, counterbalanced, within-subject field study with N = 24 participants to assess the impact of these artifacts on physical activity, reflection, and motivation. We collected quantitative data on physical activity and administered daily and weekly questionnaires, employing individual Likert items and standardized instruments, as well as conducted interviews post-prototype usage. Our findings indicate that while both artifacts act as reminders for physical activity, the physical artifact was superior in terms of user engagement. The study revealed that this can be attributed to the higher perceived presence and, thereby, enhanced social interaction, which acts as a motivational source for activity. In this sense, situated artifacts gently nudge toward sustainable health behavior change.

Augmented Reality (AR) is promising in providing individual learning support for students.However, it is currently unknown which display technology is appropriate to use in classroom settings.In this work, we investigate different AR displays and their usability

We present a sensory substitution-based method for representing locations of remote objects in 3D space via haptics. By imitating auditory localization processes, we enable vibrotactile localization abilities similar to those of some spiders, elephants, and other species. We evaluated this concept in virtual reality by modulating the vibration amplitude of two controllers depending on relative locations to a target. We developed two implementations applying this method using either ear or hand locations. A proof-of-concept study assessed localization performance and user experience, achieving under 30° differentiation between horizontal targets with no prior training. This unique approach enables localization by using only two actuators, requires low computational power, and could potentially assist users in gaining spatial awareness in challenging environments. We compare the implementations and discuss the use of hands as ears in motion, a novel technique not previously explored in the sensory substitution literature.

Gait is a distinctive behavioral trait, yet its vulnerability against imitation remains underexplored in immersive environments. We present a study investigating how real-time visual feedback in virtual reality (VR) influences a person’s ability to mimic another’s gait. Through two experiments, we first identify the most usable feedback design (N = 8), then evaluate its impact on imitation performance compared to a baseline without feedback (N = 18). We analyze positional and rotational similarity between participants and target avatars, examining the influence of avatar–user gender matching and repeated practice. Our findings reveal that visual feedback significantly improves rotational alignment and that practice leads to measurable improvements in mimicry accuracy. We discuss implications for avatar embodiment, personalization in VR applications, and potential considerations for behavioral biometric systems. We also contribute a publicly available dataset of gait mimicry in VR, supporting further research on motion learning and imitation.

Spatial Augmented Reality (SAR) can enhance learning experiences through interactive, real-time digital information overlays. Using SAR, content can be projected directly onto physical paper to provide students with in situ task feedback. Our work explores how students perceive different types of SAR feedback. We first identified feedback methods and dimensions from a literature review. We then conducted an expert focus group (N = 5) of professionals who had backgrounds in education and teaching experience. With the focus group, we aimed to expand on the literature review results to identify feedback modalities and dimensions commonly used in classrooms today. Next, we performed a field study (N = 16) with high school students in which we compared the perception of different feedback modalities (text, image, video) and familiarity (neutral, unfamiliar, familiar) in a classroom setting. Our results revealed that perceived user distraction and novelty are significantly affected by feedback modality through a large effect, with videos being perceived as more distracting and more novel. Familiar, trusted individuals best deliver positive feedback, whereas negative feedback from people should be avoided. We discuss the usage of feedback modalities in various contexts, providing a foundation for future use of SAR feedback in education.

Behavioral Biometrics in Virtual Reality (VR) allow for implicit user identification, as the head- and hand-movements that can be captured from the head-mounted display and the controllers are highly descriptive of the user’s true identity. Such body movements have been explored in the past; however, to date, it is unclear how they perform in settings where more than one person interacts in a shared virtual environment. In this work, we explored through a user study (N=40) how behavioral biometrics in VR change when one or more persons interact with each other in a shared virtual environment and whether this is influenced by the nature of the interaction itself. We find that user identification is possible with up to 83.38 % by applying deep learning models, and that particularly cooperative interactions between multiple VR users lead to highly identifiable body movements. Our results help in advancing behavioral biometrics for seamless user identification in VR, as a viable alternative to using PINs and passwords.

Dynamic Point Clouds are a representation for three-dimensional (3D) immersive media that allows users to freely navigate a scene while consuming the content. However, this comes at the cost of substantial data size, requiring efficient compression techniques to make point cloud videos accessible. Addressing this, Video-based Point Cloud Compression (V-PCC) projects points into 2D patches to compress video frames, leveraging the high compression efficiency of legacy video codecs and exploiting temporal correlations in the two-dimensional (2D) images. However, clustering and projecting points into meaningful 2D patches is computationally intensive, leading to high encoding latency in V-PCC. Applying adaptive streaming techniques, originating from traditional video streaming, multiplies the computational effort as multiple encodings of the same content are required. In this light, transcoding a compressed representation into lower qualities for dynamic adaptation to user requirements is gaining popularity. To address the high latency when employing the full decoder-encoder stack of V-PCC during transcoding, we propose RABBIT, a novel technique that only re-encodes the underlying video sub-streams. This is in contrast to slow V-PCC transcoding that reconstructs and re-encodes the raw point cloud at a new quality setting. By eliminating expensive overhead resulting from calculations based on the 3D space representation, the latency of RABBIT is bounded by the latency of transcoding the underlying video streams, allowing optimized video codec implementations to be used to meet the real-time requirements of adaptive streaming systems. Our evaluations of RABBIT, using various optimized video codec implementations, shows on-par quality with the baseline V-PCC transcoding given a high-quality representation. Given unicast or multicast distribution of a point cloud stream and in-network or edge transcoders, our evaluations show the tradeoff between rate-distortion performance and the required network bandwidth.

Electrical Muscle Stimulation (EMS) has unique capabilities that can manipulate users’ actions or perceptions, such as actuating user movement while walking, changing the perceived texture of food, and guiding movements for a user learning an instrument. These applications highlight the potential utility of EMS, but such benefits may be lost if users reject EMS. To investigate user acceptance of EMS, we conducted an online survey (N = 101). We compared eight scenarios, six from HCI research applications and two from the sports and health domain. To gain further insights, we conducted in-depth interviews with a subset of the survey respondents (N = 10). The results point to the challenges and potential of EMS regarding social and technological acceptance, showing that there is greater acceptance of applications that manipulate action than those that manipulate perception. The interviews revealed safety concerns and user expectations for the design and functionality of future EMS applications.

Behavioral Biometrics in Virtual Reality (VR) enable implicit user identification by leveraging the motion data of users’ heads and hands from their interactions in VR. This spatiotemporal data forms a Kinetic Signature, which is a user-dependent behavioral biometric trait. Although kinetic signatures have been widely used in recent research, the factors contributing to their degree of identifiability remain mostly unexplored. Drawing from existing literature, this work systematically examines the influence of static and dynamic components in human motion. We conducted a user study (N = 24) with two sessions to reidentify users across different VR sports and exercises after one week. We found that the identifiability of a kinetic signature depends on its inherent static and dynamic factors, with the best combination allowing for 90.91% identification accuracy after one week had passed. Therefore, this work lays a foundation for designing and refining movement-based identification protocols in immersive environments.

There is an endeavor in the Human-Computer Interaction (HCI) community to create novel authentication schemes so that passwords finally become obsolete and practical security is enhanced. For this purpose, researchers combine behavioral biometrics with deep learning. However, because the process of creating neural networks is inherently complex and each model architecture has certain limitations, implementing research prototypes is a time-consuming and challenging task. Therefore, we present useckit, an open-source toolkit that provides deep learning algorithms to support the creation of scientific evaluations in authentication research. Useckit provides multiple paradigms for implementing user verification and identification, functions to calculate common metrics, and neural network architectures founded in literature. It is written in Python and supports researchers and practitioners in creating, implementing, and rigorously evaluating novel deep learning-based authentication schemes.

With increasing possibilities for gathering and analyzing data, decision support systems are being offered to users for various decisionmaking tasks.Previous research has shown the importance of user's trust and interaction with a system for the acceptance of system recommendations.This study summarizes current literature on user trust in decision support and explores the effect of input control and decision criteria control on user trust.Thus, we conducted a 2x2 user study with 32 material planners and maintenance planners working of an automotive manufacturer.We studied the effect of low and high input control as well as low and high criteria control on user trust and task completion time in a spare parts inventory control tool.Our findings show that while input and criteria control increased task completion time due to interactions with the system, it had no significant impact on user trust.Our findings show a need for further exploration of the impact of user control on expert users.

Remote tutoring has gained significant traction due to technological advances, primarily relying on video-conferencing tools. However, these tools are not specifically designed for tutoring. Positive tutoring experiences rely on interaction with and immediate feedback from the tutor. Moreover, traditional methods like writing, reading, and drawing in physical spaces enhance learning outcomes. Integrating these methods with physical materials and digital remote guidance could improve remote learning experiences. A technology that enables this integration is spatial augmented reality (SAR), which utilizes projection to integrate digital content into the physical world. This work introduces a Spatial Augmented Reality (SAR) remote tutoring tool that enables augmented annotations and projected video streams. We conducted a between-subject lab study (N=18) comparing learning experiences in remote tutoring between standard video conferencing tools and the introduced Spatial Augmented Reality (SAR) tool. Qualitative analysis revealed the Spatial Augmented Reality (SAR) system's benefits over video conferencing, specifically immediate feedback, in-situ annotations, improved interactivity, and enhanced social presence.

Nowadays, Augmented and Virtual Reality devices are widely available and are often shared among users due to their high cost. Thus, distinguishing users to offer personalized experiences is essential. However, currently used explicit user authentication (e.g., entering a password) is tedious and vulnerable to attack. Therefore, this work investigates the feasibility of implicitly identifying users by their hand tracking data. In particular, we identify users by their uni- and bimanual finger behavior gathered from their interaction with eight different universal interface elements, such as buttons and sliders. In two sessions, we recorded the tracking data of 16 participants while they interacted with various interface elements in Augmented and Virtual Reality. We found that user identification is possible with up to 95% accuracy across sessions using an explainable machine learning approach. We conclude our work by discussing differences between interface elements, and feature importance to provide implications for behavioral biometric systems.A video abstract of this work is available online at: https://identifying-users-by-hand-tracking-data.hcigroup.de

Scanning everyday objects with depth sensors is the state-of-the-art approach to generating point clouds for realistic 3D representations. However, the resulting point cloud data suffers from outliers and contains irrelevant data from neighboring objects. To obtain only the desired 3D representation, additional manual segmentation steps are required. In this paper, we compare three different technology classes as independent variables (desktop vs. tablet vs. virtual reality) in a within-subject user study (N = 18) to understand their effectiveness and efficiency for such segmentation tasks. We found that desktop and tablet still outperform virtual reality regarding task completion times, while we could not find a significant difference between them in the effectiveness of the segmentation. In the post hoc interviews, participants preferred the desktop due to its familiarity and temporal efficiency and virtual reality due to its given three-dimensional representation.

Virtual mirrors displaying weight changes can support users in forming healthier habits by visualizing potential future body shapes. However, these often come with privacy, feasibility, and cost limitations. This paper introduces the Magic Mirror, a novel distortion-based mirror that leverages curvature effects to alter the appearance of body size while preserving privacy. We constructed the Magic Mirror and compared it to a video-based alternative. In an online study ( N =115), we determined the optimal parameters for each system, comparing weight change visualizations and manipulation levels. Afterward, we conducted a laboratory study ( N =24) to compare the two systems in terms of user perception, motivational potential, and willingness to use daily. Our findings indicate that the Magic Mirror surpasses the video-based mirror in terms of suitability for residential application, as it addresses feasibility concerns commonly associated with virtual mirrors. Our work demonstrates that mirrors that display weight changes can be implemented in users’ homes without any cameras, ensuring privacy.

Robots are becoming increasingly omnipresent in our daily lives, supporting us and carrying out autonomous tasks. In Human-Robot Interaction, human actors benefit from understanding the robot’s motion intent to avoid task failures and foster collaboration. Finding effective ways to communicate this intent to users has recently received increased research interest. However, no common language has been established to systematize robot motion intent. This work presents a scoping review aimed at unifying existing knowledge. Based on our analysis, we present an intent communication model that depicts the relationship between robot and human through different intent dimensions (intent type, intent information, intent location). We discuss these different intent dimensions and their interrelationships with different kinds of robots and human roles. Throughout our analysis, we classify the existing research literature along our intent communication model, allowing us to identify key patterns and possible directions for future research.

We propose a novel method for seamlessly identifying users by combining thermal and visible feet features. While it is known that users’ feet have unique characteristics, these have so far been underutilized for biometric identification, as observing those features often requires the removal of shoes and socks. As thermal cameras are becoming ubiquitous, we foresee a new form of identification, using feet features and heat traces to reconstruct the footprint even while wearing shoes or socks. We collected a dataset of users’ feet (N = 21), wearing three types of footwear (personal shoes, standard shoes, and socks) on three floor types (carpet, laminate, and linoleum). By combining visual and thermal features, an AUC between 91.1% and 98.9%, depending on floor type and shoe type can be achieved, with personal shoes on linoleum floor performing best. Our findings demonstrate the potential of thermal imaging for continuous and unobtrusive user identification.

This paper analyses Human-Computer Interaction (HCI) literature reviews to provide a clear conceptual basis for authors, reviewers, and readers. HCI is multidisciplinary and various types of literature reviews exist, from systematic to critical reviews in the style of essays. Yet, there is insufficient consensus of what to expect of literature reviews in HCI. Thus, a shared understanding of literature reviews and clear terminology is needed to plan, evaluate, and use literature reviews, and to further improve review methodology. We analysed 189 literature reviews published at all SIGCHI conferences and ACM Transactions on Computer-Human Interaction (TOCHI) up until August 2022. We report on the main dimensions of variation: (i) contribution types and topics; and (ii) structure and methodologies applied. We identify gaps and trends to inform future meta work in HCI and provide a starting point on how to move towards a more comprehensive terminology system of literature reviews in HCI.

Point clouds are a mature representation format for volumetric objects in 6 degrees-of-freedom multimedia streaming. To handle the massive size of point cloud data for visually satisfying immersive media, MPEG standardized Video-based Point Cloud Compression (V-PCC), leveraging existing video codecs to achieve high compression ratios. A major challenge of V-PCC is the high encoding latency, which results in fallback solutions that exchange the compression ratio for faster point cloud codecs. This encoding effort rises significantly in adaptive streaming systems, where heterogeneous user requirements translate into a set of quality representations of the media.

Efficient spare parts management for production asset maintenance is challenging due to the volatile nature of spare part demands. Additionally, procurement of industrial spare parts is becoming more unstable with fluctuating lead times for materials. Therefore, material planners continuously need to react to changes in the supply and demand of spare parts during spare parts replenishment. A recommendation system for spare parts replenishment could support material planners in decision-making in an uncertain environment. Recommendation systems require user trust to reach user acceptance. This paper suggests a design for a prototypical application for a recommendation system, focusing on system transparency and user control. The design is developed in close collaboration with material planners in contextual interviews, and focus groups. The prototype is evaluated in a small usability study in terms of usability and user trust. Our findings show a high emphasis of users on understanding system recommendations.

Robots play a vital role in modern automation, with applications in manufacturing and healthcare. Collaborative robots integrate human and robot movements. Therefore, it is essential to ensure that interactions involve qualified, and thus identified, individuals. This study delves into a new approach: identifying individuals through robot arm movements. Different from previous methods, users guide the robot, and the robot senses the movements via joint sensors. We asked 18 participants to perform six gestures, revealing the potential use as unique behavioral traits or biometrics, achieving F1-score up to 0.87, which suggests direct robot interactions as a promising avenue for implicit and explicit user identification.

Behavioral biometrics has recently become a viable alternative method for user identification in Virtual Reality (VR). Its ability to identify users based solely on their implicit interaction allows for high usability and removes the burden commonly associated with security mechanisms. However, little is known about the temporal stability of behavior (i.e., how behavior changes over time), as most previous works were evaluated in highly controlled lab environments over short periods. In this work, we present findings obtained from a remote field study (N = 15) that elicited data over a period of eight weeks from a popular VR game. We found that there are changes in people’s behavior over time, but that two-session identification still is possible with a mean F1-score of up to 71%, while an initial training yields 86%. However, we also see that performance can drop by up to over 50 percentage points when testing with later sessions, compared to the first session, particularly for smaller groups. Thus, our findings indicate that the use of behavioral biometrics in VR is convenient for the user and practical with regard to changing behavior and also reliable regarding behavioral variation.

The global COVID-19 pandemic created a constant need for hand disinfection. While it is still essential, disinfection use is declining with the decrease in perceived personal risk (e.g., as a result of vaccination). Thus this work explores using different visual cues to act as reminders for hand disinfection. We investigated different public display designs using (1) paper-based only, adding (2) screen-based, or (3) projection-based visual cues. To gain insights into these designs, we conducted semi-structured interviews with passersby (N=30). Our results show that the screen- and projection-based conditions were perceived as more engaging. Furthermore, we conclude that the disinfection process consists of four steps that can be supported: drawing attention to the disinfection station, supporting the (subconscious) understanding of the interaction, motivating hand disinfection, and performing the action itself. We conclude with design implications for technology-supported disinfection.

In this work, we explore attacker behavior during shoulder surfing. As such behavior is often opportunistic and difficult to observe in real world settings, we leverage the capabilities of virtual reality (VR). We recruited 24 participants and observed their behavior in two virtual waiting scenarios: at a bus stop and in an open office space. In both scenarios, participants shoulder surfed private screens displaying different types of content. From the results we derive an understanding of factors influencing shoulder surfing behavior, reveal common attack patterns, and sketch a behavioral shoulder surfing model. Our work suggests directions for future research on shoulder surfing and can serve as a basis for creating novel approaches to mitigate shoulder surfing.

Cross-reality systems empower users to transition along the reality-virtuality continuum or collaborate with others experiencing different manifestations of it. However, prototyping these systems is challenging, as it requires sophisticated technical skills, time, and often expensive hardware. We present VRception, a concept and toolkit for quick and easy prototyping of cross-reality systems. By simulating all levels of the reality-virtuality continuum entirely in Virtual Reality, our concept overcomes the asynchronicity of realities, eliminating technical obstacles. Our VRception Toolkit leverages this concept to allow rapid prototyping of cross-reality systems and easy remixing of elements from all continuum levels. We replicated six cross-reality papers using our toolkit and presented them to their authors. Interviews with them revealed that our toolkit sufficiently replicates their core functionalities and allows quick iterations. Additionally, remote participants used our toolkit in pairs to collaboratively implement prototypes in about eight minutes that they would have otherwise expected to take days.

Inertial sensors integrated into smartphones provide a unique opportunity for implicitly identifying users through their gait. However, researchers identified different external factors influencing the user's gait and consequently impact gait-based user identification algorithms. While these previous studies provide important insights, a holistic comparison of external factors influencing identification algorithms is still missing. In this explorative work, we conducted a focus group with participants from biometrics research to collect and classify these factors. Next, we recorded the gait of 12 participants walking regularly and being influenced by eleven different external factors (e.g., shoes and floor types) in two separate sessions. We used a Deep Learning (DL) identification algorithm for analysis and validated the analysis results using within- and between- sessions data. We propose a categorization of gait covariates based on users' control levels. Floor types have the most significant impact on recognition accuracy. Finally, between-session analysis shows less accurate yet more robust results than within-session validation and testing.

The last decade has witnessed an increase in Electrical Muscle Stimulation (EMS) research. Myriads of application scenarios have been proposed and toolkits, as well as procedures, have been presented to the research community. While these individual explorations provide a benefit to the research community, the overarching view on the topic is still unclear which helps to understand how such technology can be used in the future. To tackle this, we conduct a systematic literature survey of research utilizing EMS as a wearable output method. We investigate the published research (N = 151) by analyzing the EMS systems’ setup, calibration, and evaluation. We create a taxonomy of the application scenarios that categorize them along two dimensions (i.e., (1) action <> perception and (2) augmentation <> induction). Further, we distill open research challenges from our results that can help to stimulate and guide research on EMS in the future.

Displays are a ubiquitous part of our everyday life. Digital, as well as paper-based displays, are used on various occasions to present us with different information. While users cannot interact with many of these displays yet, they could benefit from making them interactive. Research shows that thermal cameras can be exploited to make surfaces interactive by sensing touchpoints based on the heat signature and residue. However, most of the research done uses a stationary setup. This work proposes HeatGoggles, a wearable gadget detecting touch input on arbitrary surfaces using thermal imaging. We report on its design, prototypical implementation, and first evaluation. We show that it can be used on non-touch-enabled displays as well as other traditional information displays.

Nowadays, deep learning models enable numerous safety-critical applications, such as biometric authentication, medical diagnosis support, and self-driving cars. However, previous studies have frequently demonstrated that these models are attackable through slight modifications of their inputs, so-called adversarial attacks. Hence, researchers proposed investigating examples of these attacks with explainable artificial intelligence to understand them better. In this line, we developed an expert tool to explore adversarial attacks and defenses against them. To demonstrate the capabilities of our visualization tool, we worked with the publicly available CIFAR-10 dataset and generated one-pixel attacks. After that, we conducted an online evaluation with 16 experts. We found that our tool is usable and practical, providing evidence that it can support understanding, explaining, and preventing adversarial examples.

Modern ubiquitous computing environments are increasingly populated with smart devices that need to know the identity of users interacting with them. At the same time, the number of authentications that a user needs to perform increases, as nowadays devices such as smart TVs require authentication which was not the case in earlier times. Even for single-person households, the need to authenticate against present smart devices in the environment appears at regular intervals, ranging from TVs to voice assistants, to gaming consoles. To reduce the need for repeated authentication, we explore the concept of a system that allows the sharing of users’ authenticated identity information between smart devices, similar to the concept of Single-Sign-On on the internet. Following a preliminary field study, we show that such a system can decrease the number of necessary authentications in a ubiquitous computing environment by 84.4%, increasing usability and security.

Augmented Reality (AR) technology enables users to superpose virtual content onto their environments. However, interacting with virtual content while mobile often requires users to perform interactions in mid-air, resulting in a lack of haptic feedback. Hence, in this work, we present the ARm Haptics system, which is worn on the user's forearm and provides 3D-printed input modules, each representing well-known interaction components such as buttons, sliders, and rotary knobs. These modules can be changed quickly, thus allowing users to adapt them to their current use case. After an iterative development of our system, which involved a focus group with HCI researchers, we conducted a user study to compare the ARm Haptics system to hand-tracking-based interaction in mid-air (baseline). Our findings show that using our system results in significantly lower error rates for slider and rotary input. Moreover, use of the ARm Haptics system results in significantly higher pragmatic quality and lower effort, frustration, and physical demand. Following our findings, we discuss opportunities for haptics worn on the forearm.

This paper contributes to our understanding of user-centered attacks on smartphones. In particular, we investigate the likelihood of so-called shoulder surfing attacks during touch-based unlock events and provide insights into users' views and perceptions. To do so, we ran a two-week in-the-wild study (N=12) in which we recorded images with a 180-degree field of view lens that was mounted on the smartphone's front-facing camera. In addition, we collected contextual information and allowed participants to assess the situation. We found that only a small fraction of shoulder surfing incidents that occur during authentication are actually perceived as threatening. Furthermore, our findings suggest that our notions of (un)safe places need to be rethought. Our work is complemented by a discussion of implications for future user-centered attack-aware systems. This work can serve as a basis for usable security researchers to better design systems against user-centered attacks.

One approach to mitigate shoulder surfing attacks on mobile devices is to detect the presence of a bystander using the phone’s front-facing camera. However, a person’s face in the camera’s field of view does not always indicate an attack. To overcome this limitation, in a novel data collection study (N=16), we analysed the influence of three viewing angles and four distances on the success of shoulder surfing attacks. In contrast to prior works that mainly focused on user authentication, we investigated three common types of content susceptible to shoulder surfing: text, photos, and PIN authentications. We show that the vulnerability of text and photos depends on the observer’s location relative to the device, while PIN authentications are vulnerable independent of the observation location. We then present PrivacyScout – a novel method that predicts the shoulder-surfing risk based on visual features extracted from the observer’s face as captured by the front-facing camera. Finally, evaluations from our data collection study demonstrate our method’s feasibility to assess the risk of a shoulder surfing attack more accurately.

Virtual Reality (VR) is becoming increasingly popular both in the entertainment and professional domains. Behavioral biometrics have recently been investigated as a means to continuously and implicitly identify users in VR. Applications in VR can specifically benefit from this, for example, to adapt virtual environments and user interfaces as well as to authenticate users. In this work, we conduct a lab study (N = 16) to explore how accurately users can be identified during two task-driven scenarios based on their spatial movement. We show that an identification accuracy of up to 90% is possible across sessions recorded on different days. Moreover, we investigate the role of users’ physiology in behavioral biometrics by virtually altering and normalizing their body proportions. We find that body normalization in general increases the identification rate, in some cases by up to 38%; hence, it improves the performance of identification systems.

We envision the human body as a platform for fast take-off and landing of drones in entertainment and professional uses such as medical emergencies, rescue missions, or supporting police units. This new interaction modality challenges our knowledge of human-drone experiences, in which interaction usually occurs at a distance from the body. This work explores important factors for understanding the interplay between drones and humans. We first investigated the suitability of various body locations for landing in an online study (N = 159). Our results, presented as body maps, show that the hand and upper back are particularly well-suited body locations. We further tested these findings in a follow-up study (N = 12), in which participants experienced drones landing on their bodies through carefully designed and pre-recorded 360°videos. This immersion into the landing scenarios helped us to identify common themes and research approaches for different body parts. Taken together, the findings provide first insights into location preferences and themes for drones landing on the human body.

In automated vehicles, it is essential to feedforward motion intentions to users so that they understand the vehicle’s actions. Otherwise, non-transparency limits situation awareness and leads to mistrust. In this work, we are communicating the vehicle’s actions to the user either by displaying icons (planar HUD) or through augmented reality (contact-analog HUD) to increase transparency. We developed both concepts in a user-centered design process. Further, we evaluated them in two subsequent user studies (N = 27). In the first study, we focused on UX and trust in higher automation levels (cf. SAE level 3-5). In the second study, we focused on safety and error prevention in lower automation levels (cf. SAE levels 1-2). Our results show that both visualizations increase UX and trust in an automated system. Nevertheless, the AR approach outperforms the icon-based approach by achieving higher user experience as well as faster and less error-prone take-overs of participants.

Shoulder surfing is an omnipresent risk for smartphone users. However, investigating these attacks in the wild is difficult because of either privacy concerns, lack of consent, or the fact that asking for consent would influence people’s behavior (e.g., they could try to avoid looking at smartphones). Thus, we propose utilizing 360-degree videos in Virtual Reality (VR), recorded in staged real-life situations on public transport. Despite differences between perceiving videos in VR and experiencing real-world situations, we believe this approach to allow novel insights on observers’ tendency to shoulder surf another person’s phone authentication and interaction to be gained. By conducting a study (N=16), we demonstrate that a better understanding of shoulder surfers’ behavior can be obtained by analyzing gaze data during video watching and comparing it to post-hoc interview responses. On average, participants looked at the phone for about 11% of the time it was visible and could remember half of the applications used.

Over recent decades, with the fast growth of urbanization, new health-related issues have resulted from the changed lifestyles. People have become more sedentary. They suffer from fully-packed calendars, which does not give them enough time to care for their physical and psychological well-being. While this is a growing challenge, advancements made in internet of things (IoT) devices in the private context offer the chance to develop applications that tackle these issues. While many such applications have been envisioned, the user’s views and requirements for them remain unclear. To better understand this, we conducted a literature review in which we identified the four most common health-related scenarios in which such technologies are used. Next, we conducted an online survey (N=80), in which participants envisioned how smart home devices could realize these scenarios. Based on the results, we derive design implications that show that users value a high degree of customization and the ability to select a specific automation level through an all-in-one, easy-to-use platform.

Virtual Reality (VR) has made its way into everyday life. While VR delivers an ever-increasing level of immersion, controls and their haptics are still limited. Current VR headsets come with dedicated controllers that are used to control every virtual interface element. However, the controller input mostly differs from the virtual interface. This reduces immersion. To provide a more realistic input, we present Flyables, a toolkit that provides matching haptics for virtual user interface elements using quadcopters. We took five common virtual UI elements and built their physical counterparts. We attached them to quadcopters to deliver on-demand haptic feedback. In a user study, we compared Flyables to controller-based VR input. While controllers still outperform Flyables in terms of precision and task completion time, we found that Flyables present a more natural and playful way to interact with VR environments. Based on the results from the study, we outline research challenges that could improve interaction with Flyables in the future.

Identifying users of a Virtual Reality (VR) headset provides designers of VR content with the opportunity to adapt the user interface, set user-specific preferences, or adjust the level of difficulty either for games or training applications. While most identification methods currently rely on explicit input, implicit user identification is less disruptive and does not impact the immersion of the users. In this work, we introduce a biometric identification system that employs the user’s gaze behavior as a unique, individual characteristic. In particular, we focus on the user’s gaze behavior and head orientation while following a moving stimulus. We verify our approach in a user study. A hybrid post-hoc analysis results in an identification accuracy of up to 75 % for an explainable machine learning algorithm and up to 100 % for a deep learning approach. We conclude with discussing application scenarios in which our approach can be used to implicitly identify users.

Automated vehicles (AVs) are on the edge of being available on the mass market. Research often focuses on technical aspects of automation, such as computer vision, sensing, or artificial intelligence. Nevertheless, researchers also identified several challenges from a human perspective that need to be considered for a successful introduction of these technologies. In this paper, we first analyze human needs and system acceptance in the context of AVs. Then, based on a literature review, we provide a summary of current research on in-car driver-vehicle interaction and related human factor issues. This work helps researchers, designers, and practitioners to get an overview of the current state of the art.

Understanding user needs and behavior in automated vehicles (AVs) while traveling is essential for future in-vehicle interface and service design. Since AVs are not yet market-ready, current knowledge about AV use and perception is based on observations in other transportation modes, interviews, or surveys about the hypothetical situation. In this paper, we close this gap by presenting real-world insights into the attitude towards highly automated driving and non-driving-related activities (NDRAs). Using a Wizard of Oz AV, we conducted a real-world driving study (N = 12) with six rides per participant during multiple days. We provide insights into the users’ perceptions and behavior. We found that (1) the users’ trust a human driver more than a system, (2) safety is the main acceptance factor, and (3) the most popular NDRAs were being idle and the use of the smartphone.

One of the main benefits of interactive Virtual Reality (VR) applications is that they provide a high sense of immersion. As a result, users lose their sense of real-world space which makes them vulnerable to collisions with real-world objects. In this work, we propose a novel approach to prevent such collisions using Electrical Muscle Stimulation (EMS). EMS actively prevents the movement that would result in a collision by actuating the antagonist muscle. We report on a user study comparing our approach to the commonly used feedback modalities: audio, visual, and vibro-tactile. Our results show that EMS is a promising modality for restraining user movement and, at the same time, rated best in terms of user experience.

Human computer interaction models deliver a conceptual overview of the fundamental communication processes between the computer and the human. The aim of these models is to provide an easy understanding of a system’s behaviour on a meta level. Existing HCI models cover most of the actual interaction modalities. New arising technologies provide the possibility of directly manipulating the human action – called actuation. Such interaction, however, is not covered by classical models. In this work, we propose additional relations extending current HCI models to include novel interactions using human actuation.

Winter sports like skiing are becoming increasingly popular for both competitive and recreational activities. To minimize the risk of injury, new innovations in skiing equipment have been developed in recent years. However, unexpected slope conditions can still increase risks during skiing. The static categorisation of ski slopes in winter sports resorts does not take into account dynamic changes of difficulty due to high traffic volumes or sudden weather changes. Up to now, efforts have been made to measure the current conditions via satellite imaging or installations on the slope. However, this requires intervention in nature and causes high maintenance costs. To solve these issues we present our preliminary design of a wearable system to let skiers implicitly measure current slope conditions during their skiing experience. Audio and motion data are recorded from a prototype mounted on a ski boot. We show that the data generated by the prototype can be successfully classified with a neural network. We collected data from a skiing activity to demonstrate our concept and discuss the identified challenges in fitting the proposed approach to winter sports equipment.

With the rise of the Internet of Things, home appliances become connected and they can proactively provide status information to users. Facing a steadily increasing number of notification sources, it is unclear how information from smart home devices should be provided without overloading the users’ attention. In this paper, we investigate the design of non-urgent smart home notifications using a smart plant system. Based on feedback from focus groups, we designed four notification types and compared them in an eight-week in-situ study. We show that notifications displayed on smart home devices are preferred to those received on smartphones. Event-based notifications are unobtrusive, actionable and are preferred to persistent notifications. We derive guidelines that address the need of being in control, opportune locations for notification delivery at opportune moments, notification blindness, the importance of discretizing continuous information, and combining related notifications.

Self-driving cars will relief the human from the driving task. Nevertheless, the human might want to intervene in the driving process and thus needs the possibility to control the car. Switching back to fully manual controls is uncomfortable once being passive and engaging in non-driving-related activities. A more comfortable way is controlling the car with elemental maneuvers (e.g., "turn left" or "stop"). Whereas touch interaction concepts exist, contactless interaction through voice and mid-air gestures has not yet been explored for maneuver-based car control. In this paper, we, therefore, compare the general eligibility of voice and mid-air gesture with touch interaction as the primary maneuver selection mechanism in a driving simulator study. Our results show high usability for all modalities. Contactless interaction leads to a more positive emotional perception of the interaction, yet mid-air gestures lead to higher task load. Overall, voice and touch control are preferred over mid-air gestures by most users.

Virtual worlds are infinite environments in which the user can move around freely. When shifting from controller-based movement to regular walking as an input, the limitation of the real world also limits the virtual world. Tackling this challenge, we propose the use of electrical muscle stimulation to limit the necessary real-world space to create an unlimited walking experience. We thereby actuate the users` legs in a way that they deviate from their straight route and thus, walk in circles in the real world while still walking straight in the virtual world. We report on a study comparing this approach to vision shift - the state of the art approach - as well as combining both approaches. The results show that particularly combining both approaches yield high potential to create an infinite walking experience.

Since the release of the first activity tracker, there has been a steady increase in the number of sensors embedded in wearable devices and with it in the amount and diversity of information that can be derived from these sensors. This development leads to novel privacy threats for users. In a web survey with 248 participants, we explored whether users' willingness to share private data is dependent on how the data is requested by an application. Specifically, requests can be formulated as access to sensor data or as access to information derived from the sensor data (e.g., accelerometer vs. sleep quality). We show that non-expert users lack an understanding of how the two representation levels relate to each other. The results suggest that the willingness to share sensor data over derived information is governed by whether the derived information has positive or negative connotations (e.g., training intensity vs. life expectancy). Using the results of the survey, we derive implications for supporting users in protecting their private data collected via wearable sensors.

The uprising levels of autonomous vehicles allow the drivers to shift their attention to non-driving tasks while driving (i.e., texting, reading, or watching movies). However, these systems are prone to failure and, thus, depending on human intervention becomes crucial in critical situations. In this work, we propose using human actuation as a new mean of communicating take-over requests (TOR) through proprioception. We conducted a user study via a driving simulation in the presence of a complex working memory span task. We communicated TORs through four different modalities, namely, vibrotactile, audio, visual, and proprioception. Our results show that the vibrotactile condition yielded the fastest reaction time followed by proprioception. Additionally, proprioceptive cues resulted in the second best performance of the non-driving task following auditory cues.

Currently offered autonomous vehicles still require the human intervention. For instance, when the system fails to perform as expected or adapts to unanticipated situations. Given that reliability of autonomous systems can fluctuate across conditions, this work is a first step towards understanding how this information ought to be communicated to users. We conducted a user study to investigate the effect of communicating the system's reliability through a feedback bar. Subjective feedback was solicited from participants with questionnaires and semi-structured interviews. Based on the qualitative results, we derived guidelines that serve as a foundation for the design of how autonomous systems could provide continuous feedback on their reliability.

Current navigation systems for motor cyclists use visual or auditory cues for guidance. However, this poses a challenge to the motorcyclists since their visual and auditory channels are already occupied with controlling the motorbike, paying attention to other road users, and planing the next turn. In this work, we explore how tactile feedback can be used to guide motorcyclists. We present MOVING (MOtorbike VIbrational Navigation Guidance), a smart kidney belt that presents navigation cues through 12 vibration motors. In addition, we report on the design process of this wearable and on an evaluation with 16 participants in a real world riding setting. We show that MOVING outperforms off-the-shelf navigation systems in terms of turn errors and distraction.

We present LYRA, a modular in-flight system that enhances service and assists flight attendants during their work. LYRA enables passengers to browse and order services from their smartphones. Smart glasses and a smart shoe-clip with RFID reader module provides flight attendants with situated information. We gained first insights into how flight attendants and passengers use of the system during a long distance flight from Frankfurt to Houston.

Notifications on mobile devices are a prominent source of interruptions. Previous work suggests using opportune moments to deliver notifications to reduce negative effects. In this paper, we instead explore the manual deferral of notifications. We developed an Android app that allows users to "snooze" mobile notifications for a user-defined amount of time or to a user-defined point in time. Using this app, we conducted a year-long in-the-wild study with 295 active users. To complement the findings, we recruited 16 further participants who used the app for one week and subsequently interviewed them. In both studies, snoozing was mainly used to defer notifications related to people and events. The reasons for deferral were manifold, from not being able to attend notifications immediately to not wanting to. Daily routines played an important role in the deferral of notifications. Most notifications were deferred to the same day or next morning, and a deferral of more than two days was an exception. Based on our findings, we derive design implications that can inform the design of future smart notification systems.

Applying Augmented Reality in education is being explored by many scientists. Therefore, we augment digital slides of lectures in higher education. We implemented a web server application, which allows professors to create their own AR slides. We also developed a mobile app for students to scan the slides and view the augmentation in lectures or learning sessions. To assess the usability of our system, we conducted a study with fifteen students and two professors. Students' feedback indicated that our AR app could be integrated into education. Professors, on the other hand, reported improvement suggestions. However, both groups supported applying the system in real lectures.

Cooking has the potential to bring joy, a sense of achievement and social presence to individuals and groups. Food properties (food cues) such as scent, sound and taste are rich cues to the state of the cooking process, as well as, providing memories evoking emotions attached to social situations and people from the past. Thus, optimal methods to capture and present these cues is an on-going research challenge. In this workshop, we bring researchers, amateur cooks, and designers together to explore two research questions:1) how does enhancing our awareness of food temperature support a joyful and/or a memorable cooking experience? and 2) how can we use food cues to enhance our sense of joy, empower reminiscence and nudge communication between individuals?. We aim through this event to foster a research community that uses food interaction as a medium to promote a sense of joyfulness and social wellbeing.

With the expansion of multicultural communities, raising awareness to diminish cultural misconceptions became a must. We investigate different research probes using interactive Virtual Reality (VR) applications to increase such awareness. VR environments provide a high sense of immersion, which enhance the user's experience. In this work, we present three VR interactive research probes that allow users to understand cultural differences and to reduce cultural misconceptions.

We present VRHapticDrones, a system utilizing quadcopters as levitating haptic feedback proxy. A touchable surface is attached to the side of the quadcopters to provide unintrusive, flexible, and programmable haptic feedback in virtual reality. Since the users' sense of presence in virtual reality is a crucial factor for the overall user experience, our system simulates haptic feedback of virtual objects. Quadcopters are dynamically positioned to provide haptic feedback relative to the physical interaction space of the user. In a first user study, we demonstrate that haptic feedback provided by VRHapticDrones significantly increases users' sense of presence compared to vibrotactile controllers and interactions without additional haptic feedback. In a second user study, we explored the quality of induced feedback regarding the expected feeling of different objects. Results show that VRHapticDrones is best suited to simulate objects that are expected to feel either light-weight or have yielding surfaces. With VRHapticDrones we contribute a solution to provide unintrusive and flexible feedback as well as insights for future VR haptic feedback systems.

Since mobile interaction takes place in almost every context, shoulder surfing attacks are becoming more and more a threat to user's privacy. While several approaches exist to prevent these attacks for the authentication process, protecting the actual interaction has not yet been in the main focus of research. In this work, we present the concept of communicating shoulder surfing attacks to the user. This should create awareness on the user side and help preventing this type of privacy invasion. We present out shoulder surfer detection mobile application, called DSSytem, and report on a focus group that helped to design this system. We also report on the results of a user study in which we compare four different notification methods, namely, vibro-tactile, front LED, on-screen icon, and video preview feedback. Vibro-tactile feedback results in the lowest reaction time of the participants and is also favoured throughout the follow-up semi-structured interviews.

Smartphones enable people to learn new languages whenever and wherever they want. This popularized mobile language learning apps (MLLAs) and in particular micro learning that offers simple and short learning units to keep the user on track. Due to the ubiquitous use of these applications, they have to adapt to the users' current situation to provide an optimal learning experience. To gain insights into how users perceive common usage scenarios, we conducted an online survey (N=74) and clustered all described learning scenarios into five categories of usage situations. We outlined internal and contextual factors which are characteristic for these situations and discussed those in a follow-up focus group with HCI experts (N=4). During this focus group, we collected four design recommendations to adapt MLLAs to situations of users' (a) high attention levels, (b) tiredness or exhaustion, (c) highly demanding environments, or (d) low motivation.

In this paper we investigate how Augmented Reality (AR) technology influences children during creative content generation in playful settings. The work is motivated by the recent spread of AR and the fact that children get in touch with this technology through their smart phones very early on. To understand the consequences, we implemented an app for smart mobile devices that allows children to create treasure hunts using GPS coordinates and marker-based AR functionality. During a qualitative user study, we asked students (n=27) to create traditional (paper + art supplies) and digital (paper + art supplies + AR app) treasure hunts and compared the resulting games, among other metrics, in terms of complexity, length and types of media used. Whereas traditional treasure hunts were linear, centered around locations and delivered information with text only, digital treasure hunts were more complex, focused on visual aspects and frequently integrated storytelling.

PINs and patterns remain among the most widely used knowledge-based authentication schemes. As thermal cameras become ubiquitous and affordable, we foresee a new form of threat to user privacy on mobile devices. Thermal cameras allow performing thermal attacks, where heat traces, resulting from authentication, can be used to reconstruct passwords. In this work we investigate in details the viability of exploiting thermal imaging to infer PINs and patterns on mobile devices. We present a study (N=18) where we evaluated how properties of PINs and patterns influence their thermal attacks resistance. We found that thermal attacks are indeed viable on mobile devices; overlapping patterns significantly decrease successful thermal attack rate from 100% to 16.67%, while PINs remain vulnerable (>72% success rate) even with duplicate digits. We conclude by recommendations for users and designers of authentication schemes on how to resist thermal attacks.

The human body reveals emotional and bodily states through measurable signals, such as body language and electroencephalography. However, such manifestations are difficult to communicate to others remotely. We propose EmotionActuator, a proof-of-concept system to investigate the transmission of emotional states in which the recipient performs emotional gestures to understand and interpret the state of the sender.We call this kind of communication embodied emotional feedback, and present a prototype implementation. To realize our concept we chose four emotional states: amused, sad, angry, and neutral. We designed EmotionActuator through a series of studies to assess emotional classification via EEG, and create an EMS gesture set by comparing composed gestures from the literature to sign-language gestures. Through a final study with the end-to-end prototype interviews revealed that participants like implicit sharing of emotions and find the embodied output to be immersive, but want to have control over shared emotions and with whom. This work contributes a proof of concept system and set of design recommendations for designing embodied emotional feedback systems.

Obtaining information about audience engagement in presentations is a valuable asset for presenters in many domains. Prior literature mostly utilized explicit methods of collecting feedback which induce distractions, add workload on audience and do not provide objective information to presenters. We present EngageMeter - a system that allows fine-grained information on audience engagement to be obtained implicitly from multiple brain-computer interfaces (BCI) and to be fed back to presenters for real time and post-hoc access. Through evaluation during an HCI conference (Naudience=11, Npresenters=3) we found that EngageMeter provides value to presenters (a) in real-time, since it allows reacting to current engagement scores by changing tone or adding pauses, and (b) in post-hoc, since presenters can adjust their slides and embed extra elements. We discuss how EngageMeter can be used in collocated and distributed audience sensing as well as how it can aid presenters in long term use.

Electrical Muscle Stimulation (EMS) recently received considerable attention in the HCI community. By applying small signals to the user's body, different types of movement can be generated. These movements allow designers to create more meaningful and embodied haptic feedback compared to vibrotactile feedback. This advantage also comes with further technical and practical challenges which need to be tackled. These challenges include a fine grained calibration procedure and a close contact to the user's body at specific on-body locations. This tutorial gives an overview about current research projects, challenges, and opportunities to use EMS for providing rich embodied feedback followed by a hands on experience. The main goal of this tutorial is that participants get a basic understanding of how EMS works and how systems that are using EMS can be developed and evaluated.

Smartphones, wearables, and other mobile devices often use tactile feedback for notifying users. This feedback type proved to be beneficial since it does not occupy the visual or auditory channel. However, it still can be distracting in other situations such as when users are already stressed. To investigate tactile feedback patterns which do not increase the user's stress level, we developed two wrist-worn prototypes capable of providing tactile feedback (i.e., vibrotactile and pressure-based feedback). Further, we conducted a user-study with 14 participants comparing both feedback types. The results suggest that vibrotactile feedback increases the user's stress level more, compared to pressure-based feedback particularly applied when the user currently has a low stress level. Consequently, we present implications for designing notifications for mobile and wearable devices.

Today's workplaces are dynamic and complex. Digital data sources such as email and video conferencing aim to support workers but also add to their burden of multitasking. Psychophysiological sensors such as Electroencephalography (EEG) can provide users with cues about their cognitive state. We introduce BrainAtWork, a workplace engagement and task logger which shows users their cognitive state while working on different tasks. In a lab study with eleven participants working on their own real-world tasks, we gathered 16 hours of EEG and PC logs which were labeled into three classes: central, peripheral and meta work. We evaluated the usability of BrainAtWork via questionnaires and interviews. We investigated the correlations between measured cognitive engagement from EEG and subjective responses from experience sampling probes. Using random forests classification, we show the feasibility of automatically labeling work tasks into work classes. We discuss how BrainAtWork can support workers on the long term through encouraging reflection and helping in task scheduling.

Although interaction with computing systems has become remarkably diverse in recent years, the computer mouse remained the primary pointing device for daily computer use. Being solely an input device, the classical mouse decouples input from output. In this paper, we propose to extend the mouse to a device that can be actuated by the user and the computer. We developed a mouse that allows its position and button state to be actuated. In a technical evaluation, we test the spatial resolution of our system and how effectively feedback is communicated to the user. In a subjective assessment, we explore users' reactions to four use cases including games and office applications, highlighting the potential of the device. Through a quantitative assessment, we investigate whether perceiving the movement of the mouse helps to learn gestures. Finally, we discuss how a mouse providing feedback can be used to build novel interaction techniques.

Wearable devices like smart watches or eye-wear computers are storing a myriad of personal information. Today, wearable devices support input techniques such as speech and gesture input. These user input methods however, are not well suited for authentication. With the development of smart textiles the design space for interaction for small smart devices can be increased. In this paper, we present the concept of FabricID a system which identifies users' hand-prints with a smart textile integrated into the sleeve of the user. To evaluate our concept, we recorded hand-prints of 16 users. We classified all recorded hand-prints and received an identification rate of 82.5% for all 16 users and on average 93.62% for groups of 4 users.

Secure user identification is important for the increasing number of eyewear computers but limited input capabilities pose significant usability challenges for established knowledge-based schemes, such as passwords or PINs. We present SkullConduct, a biometric system that uses bone conduction of sound through the user's skull as well as a microphone readily integrated into many of these devices, such as Google Glass. At the core of SkullConduct is a method to analyze the characteristic frequency response created by the user's skull using a combination of Mel Frequency Cepstral Coefficient (MFCC) features as well as a computationally light-weight 1NN classifier. We report on a controlled experiment with 10 participants that shows that this frequency response is person-specific and stable -- even when taking off and putting on the device multiple times -- and thus serves as a robust biometric. We show that our method can identify users with 97.0% accuracy and authenticate them with an equal error rate of 6.9%, thereby bringing biometric user identification to eyewear computers equipped with bone conduction technology.

Cued-recall graphical passwords have a lot of potential for secure user authentication, particularly if combined with saliency masks to prevent users from selecting weak passwords. Saliency masks were shown to significantly improve password security by excluding those areas of the image that are most likely to lead to hotspots. In this paper we investigate the impact of such saliency masks on the memorability of cued-recall graphical passwords. We first conduct two pre-studies (N=52) to obtain a set of images with three different image complexities as well as real passwords. A month-long user study (N=26) revealed that there is a strong learning effect for graphical passwords, in particular if defined on images with a saliency mask. While for complex images, the learning curve is steeper than for less complex ones, they best supported memorability in the long term, most likely because they provided users more alternatives to select memorable password points. These results complement prior work on the security of such passwords and underline the potential of saliency masks as both a secure and usable improvement to cued-recall gaze-based graphical passwords.

Today, the vast majority of research on novel automotive user interface technologies is conducted in the lab, often using driving simulation. While such studies are important in early stages of the design process, we argue that ultimately studies need to be conducted in the real-world in order to investigate all aspects crucial for adoption of novel user interface technologies in commercial vehicles. In this paper, we present a case study that investigates introducing autostereoscopic 3D dashboards into cars. We report on studying this novel technology in the real world, validating and extending findings of prior simulator studies. Furthermore, we provide guidelines for practitioners and researchers to design and conduct real-world studies that minimize the risk for participants while at the same time yielding ecologically valid findings.

Distant pointing at objects and persons is a highly expressive gesture that is widely used in human communication. Pointing is also used to control a range of interactive systems. For determining where a user is pointing at, different ray casting methods have been proposed. In this paper we assess how accurately humans point over distance and how to improve it. Participants pointed at projected targets from 2m and 3m while standing and sitting. Testing three common ray casting methods, we found that even with the most accurate one the average error is 61.3cm. We found that all tested ray casting methods are affected by systematic displacements. Therefore, we trained a polynomial to compensate this displacement. We show that using a user-, pose-, and distant-independent quartic polynomial can reduce the average error by 37.3%

Pedestrian navigation systems require users to perceive, interpret, and react to navigation information. This can tax cognition as navigation information competes with information from the real world. We propose actuated navigation, a new kind of pedestrian navigation in which the user does not need to attend to the navigation task at all. An actuation signal is directly sent to the human motor system to influence walking direction. To achieve this goal we stimulate the sartorius muscle using electrical muscle stimulation. The rotation occurs during the swing phase of the leg and can easily be counteracted. The user therefore stays in control. We discuss the properties of actuated navigation and present a lab study on identifying basic parameters of the technique as well as an outdoor study in a park. The results show that our approach changes a user's walking direction by about 16°/m on average and that the system can successfully steer users in a park with crowded areas, distractions, obstacles, and uneven ground.

Common user authentication methods on smartphones, such as lock patterns, PINs, or passwords, impose a trade-off between security and password memorability. Image-based passwords were proposed as a secure and usable alternative. As of today, however, it remains unclear how such schemes are used in the wild. We present the first study to investigate how image-based passwords are used over long periods of time in the real world. Our analyses are based on data from 2318 unique devices collected over more than one year using a custom application released in the Android Play store. We present an in-depth analysis of what kind of images users select, how they define their passwords, and how secure these passwords are. Our findings provide valuable insights into real-world use of image-based passwords and inform the design of future graphical authentication schemes.

Eating is not only one of the most fundamental human needs but also among the most regular activities. Acquiring food, preparing meals, and socializing around food are deeply rooted in all human cultures. In this paper we show how food can not only serve to satisfy hunger but also become a new display technology. Through food augmentation, a dinner could communicate its ingredients, convey messages, or provide instructions such as the recipe of a meal. We show how to augment a large range of food with laser. We conducted a series of focus groups to gather people's first impressions and derive a broad range of meaningful augmentation scenarios. We discuss the perceived benefits, opportunities, and concerns. Additionally, we evaluated a number of scenarios through an online survey. The most readily accepted augmentation scenarios include adding practical information, increasing awareness about the food, and augmenting food items with a natural skin.

In this paper, we investigate user performance for stereoscopic automotive user interfaces (UI). Our work is motivated by the fact that stereoscopic displays are about to find their way into cars. Such a safety-critical application area creates an inherent need to understand how the use of stereoscopic 3D visualizations impacts user performance. We conducted a comprehensive study with 56 participants to investigate the impact of a 3D instrument cluster (IC) on primary and secondary task performance. We investigated different visualizations (2D and 3D) and complexities (low vs. high amount of details) of the IC as well as two 3D display technologies (shutter vs. autostereoscopy). As secondary tasks the participants judged spatial relations between UI elements (expected events) and reacted on pop-up instructions (unexpected events) in the IC. The results show that stereoscopy increases accuracy for expected events, decreases task completion times for unexpected tasks, and increases the attractiveness of the interface. Furthermore, we found a significant influence of the used technology, indicating that secondary task performance improves for shutter displays.

In this paper, we identify design guidelines for stereoscopic 3D (S3D) user interfaces (UIs) and present the MirrorBox and the FrameBox, two UI prototyping tools for S3D displays. As auto-stereoscopy becomes available for the mass market we believe the design of S3D UIs for devices, for example, mobile phones, public displays, or car dashboards, will rapidly gain importance. A benefit of such UIs is that they can group and structure information in a way that makes them easily perceivable for the user. For example, important information can be shown in front of less important information. This paper identifies core requirements for designing S3D UIs and derives concrete guidelines. The requirements also serve as a basis for two depth layout tools we built with the aim to overcome limitations of traditional prototyping when sketching S3D UIs. We evaluated the tools with usability experts and compared them to traditional paper prototyping.

The entertainment industry is going through a transformation, and technology development is affecting how we can enjoy and interact with the entertainment media content in new ways. In our work, we explore how to enable interaction with content in the context of 3D cinemas. This allows viewers to use their mobile phone to retrieve, for example, information on the artist of the soundtrack currently playing or a discount coupon on the watch the main actor is wearing. We are particularly interested in the user experience of the interactive 3D cinema concept, and how different interactive elements and interaction techniques are perceived. We report on the development of a prototype application utilizing smart phones and on an evaluation in a cinema context with 20 participants. Results emphasize that designing for interactive cinema experiences should drive for holistic and positive user experiences. Interactive content should be tied together with the actual video content, but integrated into contexts where it does not conflict with the immersive experience with the movie.

Thermal cameras have recently drawn the attention of HCI researchers as a new sensory system enabling novel interactive systems. They are robust to illumination changes and make it easy to separate human bodies from the image background. Far-infrared radiation, however, has another characteristic that distinguishes thermal cameras from their RGB or depth counterparts, namely thermal reflection. Common surfaces reflect thermal radiation differently than visual light and can be perfect thermal mirrors. In this paper, we show that through thermal reflection, thermal cameras can sense the space beyond their direct field-of-view. A thermal camera can sense areas besides and even behind its field-of-view through thermal reflection. We investigate how thermal reflection can increase the interaction space of projected surfaces using camera-projection systems. We moreover discuss the reflection characteristics of common surfaces in our vicinity in both the visual and thermal radiation bands. Using a proof-of-concept prototype, we demonstrate the increased interaction space for hand-held camera-projection system. Furthermore, we depict a number of promising application examples that can benefit from the thermal reflection characteristics of surfaces.

Touch-enabled user interfaces have become ubiquitous, such as on ATMs or portable devices. At the same time, authentication using touch input is problematic, since finger smudge traces may allow attackers to reconstruct passwords. We present SmudgeSafe, an authentication system that uses random geometric image transformations, such as translation, rotation, scaling, shearing, and flipping, to increase the security of cued-recall graphical passwords. We describe the design space of these transformations and report on two user studies: A lab-based security study involving 20 participants in attacking user-defined passwords, using high quality pictures of real smudge traces captured on a mobile phone display; and an in-the-field usability study with 374 participants who generated more than 130,000 logins on a mobile phone implementation of SmudgeSafe. Results show that SmudgeSafe significantly increases security compared to authentication schemes based on PINs and lock patterns, and exhibits very high learnability, efficiency, and memorability.

In this paper, we investigate the concept of gaze-based interaction with 3D user interfaces. We currently see stereo vision displays becoming ubiquitous, particularly as auto-stereoscopy enables the perception of 3D content without the use of glasses. As a result, application areas for 3D beyond entertainment in cinema or at home emerge, including work settings, mobile phones, public displays, and cars. At the same time, eye tracking is hitting the consumer market with low-cost devices. We envision eye trackers in the future to be integrated with consumer devices (laptops, mobile phones, displays), hence allowing the user's gaze to be analyzed and used as input for interactive applications. A particular challenge when applying this concept to 3D displays is that current eye trackers provide the gaze point in 2D only (x and y coordinates). In this paper, we compare the performance of two methods that use the eye's physiology for calculating the gaze point in 3D space, hence enabling gaze-based interaction with stereoscopic content. Furthermore, we provide a comparison of gaze interaction in 2D and 3D with regard to user experience and performance. Our results show that with current technology, eye tracking on stereoscopic displays is possible with similar performance as on standard 2D screens.

Calling while driving a car has become very common since the rise of mobile phones. Drivers use their phone despite the fact that calling in the car is potentially distracting and dangerous. Prohibiting communication while driving is not a good idea as there are also positive effects of calling (e.g., ability to notify about a delay, staying awake, preventing fatigue, guidance at foreign places).

Driving a car is becoming increasingly complex. Many new features (e.g., for communication or entertainment) that can be used in addition to the primary task of driving a car increase the driver's workload. Assessing the driver's workload, however, is still a challenging task. A variety of means are explored which rather focus on experimental conditions than on real world scenarios (e.g., questionnaires). We focus on physiological data that may be assessed in an non-obtrusive way in the future and is therefore applicable in the real world.

Implementing controls in the car becomes a major challenge: The use of simple physical buttons does not scale to the increased number of assistive, comfort, and infotainment functions. Current solutions include hierarchical menus and multi-functional control devices, which increase complexity and visual demand. Another option is speech control, which is not widely accepted, as it does not support visibility of actions, fine-grained feedback, and easy undo of actions. Our approach combines speech and gestures. By using speech for identification of functions, we exploit the visibility of objects in the car (e.g., mirror) and simple access to a wide range of functions equaling a very broad menu. Using gestures for manipulation (e.g., left/right), we provide fine-grained control with immediate feedback and easy undo of actions. In a user-centered process, we determined a set of user-defined gestures as well as common voice commands. For a prototype, we linked this to a car interior and driving simulator. In a study with 16 participants, we explored the impact of this form of multimodal interaction on the driving performance against a baseline using physical buttons. The results indicate that the use of speech and gesture is slower than using buttons but results in a similar driving performance. Users comment in a DALI questionnaire that the visual demand is lower when using speech and gestures.

In public places we can observe that many conventional displays are replaced by digital displays, a lot of them networked. These displays mainly show advertising in a similar way to television's commercial break not exploiting the opportunities of the new medium [1]. Several approaches of interaction between mobile devices and public displays have been investigated over the last 15 years. In this demo we concentrate on challenges that are specific to public displays used for advertising. In particular we focus on how new approaches for interaction with content, means for content creation, and tools for follow-ups can be implemented based on mobile devices. With Digifieds we present a research system that has been used to explore different research questions and to showcase the potential of interactive advertising in public space.

During the last decade, the number of functions of automotive user interfaces has increased rapidly. Besides traditional controls to drive a car, driver assistance, infotainment, entertainment, and comfort systems need to be controlled while driving. This does not only affect the driver's cognitive workload but also leads to increased complexity in designing automotive user interfaces. In this paper, we provide models and tools for rapid prototyping and the evaluation of user interfaces in this context. Usually, functional prototypes of user interfaces are implemented that allow the usability and quality to be assessed with time-consuming user studies. In contrast, in our approach we use an adapted Keystroke-Level Model (KLM) that is based on empirically collected data for typical operations in the car. It takes into account the aspect of attention switching in the car between primary tasks and other tasks. We present KLM operator times that we determined in a user study as well as a formula for estimating the task completion time. The presented model is the foundation for the MI-AUI prototyping tool that we implemented for permitting the creation of automotive interfaces using tangible controls. By demonstrating a typical operation with the MI-AUI prototype, the estimated task completion time can be calculated. MI-AUI is an evaluation tool that can be quickly and easily applied in early stages of the design process without the need to involve real drivers.