Internet memes are central to online communication, yet their visual humor is often lost in face-to-face (F2F) conversations. Augmented reality (AR) offers new ways to bring memes into F2F interactions, but it is unclear how memes can be integrated into F2F conversations using AR, and how they impact conversational dynamics. We surveyed meme users (N=29) to understand motivations and challenges in visualising memes in F2F conversations. With these insights, we developed an AR meme-sharing prototype and invited 12 pairs of friends to design AR visualizations for their memes and use them in conversations. Our analysis reveals two AR-unique visualizations: merging memes with one’s body (The-Meme-On-Me) and situating oneself in meme environment (Me-In-The-Meme). We observed two integration patterns: using speech as setup before a meme punchline, and showing memes simultaneously with speech to amplify humor. We report users’ reactions toward AR memes, showing how it enables playful social interaction.

Social VR's focus on embodied and immersive experiences has led to its increasing popularity and value for education, work, entertainment, and even our everyday social lives. However, there is also a growing concern about emerging harmful behaviours among users in social VR spaces. Despite this, existing work has highlighted a fundamental challenge in effectively identifying and mitigating such behaviours: there is no established terminology or consensus on what should be categorised as harmful behaviours in social VR. In this work, we aim to address this critical gap by providing a foundation for classifying harmful behaviours in social VR, which can help researchers, developers, and policymakers to achieve conceptual clarity and actionable insights for designing safer VR environments. Using a systematic literature review of existing work in this space, we present a comprehensive taxonomy of harmful behaviours identified in existing HCI research and classify them based on dimensions of behaviour modality (i.e., verbal, physical, environmental, and three novel system-, gesture-, and spatial-based categories) and interaction dynamics (i.e., one-to-one, one-to-many). This work provides an important conceptual contribution that enables researchers to better differentiate and systematically investigate these different types of harms in social VR. It also offers valuable guidance for the design of safer social VR environments, e.g., through informing the development of validated scales, codes of conduct, detection algorithms, and other types of interventions.

Human Motion Simulation (HMS) predicts and visualizes how virtual agents move and behave in various scenarios, aiding decision-making. However, it does not communicate the experience behind agents' behavior. To address this, we present Embodied Spatial Simulation (ESS), a concept combining augmented reality(AR) and HMS to support in-situ decision-making. It enables an XR user to create "what-if" scenarios, simulate virtual agents' behavior in situ, and experience the scenarios from their perspectives through embodiment. To demonstrate ESS, we developed a proof-of-concept prototype featuring two scenarios (e.g., fire evacuation planning). From expert walkthroughs (N=4) and a survey (N=38) with different professions, we found that incorporating embodiment into simulations can change users' intentions to use simulation outcomes. Rather than directly obtaining solutions from the results, users showed more interest in understanding the experience behind simulated behavior and using this insight to shape decisions. We discuss the potential of ESS to support future human-centered design practices.

With the recent advancements in Large Language Models (LLMs), web developers increasingly apply their code-generation capabilities to website design. However, since these models are trained on existing designerly knowledge, they may inadvertently replicate bad or even illegal practices, especially deceptive designs (DD). This paper examines whether users can accidentally create DD for a fictitious webshop using GPT-4. We recruited 20 participants, asking them to use ChatGPT to generate functionalities (product overview or checkout) and then modify these using neutral prompts to meet a business goal (e.g., “increase the likelihood of us selling our product”). We found that all 20 generated websites contained at least one DD pattern (mean: 5, max: 9), with GPT-4 providing no warnings. When reflecting on the designs, only 4 participants expressed concerns, while most considered the outcomes satisfactory and not morally problematic, despite the potential ethical and legal implications for end-users and those adopting ChatGPT’s recommendations.

Research in Augmented Reality (AR) and Virtual Reality (VR) has mostly viewed them in isolation.Yet, when used together in practical settings, AR and VR each offer unique strengths, necessitating multiple transitions to harness their advantages.This paper investigates potential challenges in Cross-Reality (CR) transitions to inform future application design.We implemented a CR system featuring a 3D modeling task that requires users to switch between PC, AR, and VR.Using a talk-aloud study (n=12) and thematic analysis, we revealed that frictions primarily arose when transitions conflicted with users' Spatial Mental Model (SMM).Furthermore, we found five transition archetypes employed to enhance productivity once an SMM was established.Our findings uncover that transitions have to focus on establishing and upholding the SMM of users across realities, by communicating differences between them.

Augmented reality displays are becoming more powerful and simultaneously more mobile. Although mobile AR is gaining popularity, it remains difficult to get an insight into users' cognitive load, despite its relevance for many mobile-based tasks. Usually, cognitive load is measured via subjective, task-disruptive self-reports such as NASA TLX. While biosensors such as galvanic skin response, heart rate variability, or pulse have been used to obtain more objective measures, these are highly susceptible to motion-induced noise. More robust techniques like EEG offer higher reliability but are impractical for mobile, real-world use. In this paper, we report on a non-contact multi-sensor approach to assess cognitive load in mobile AR. Our approach combines pupillometry, facial expression tracking, and thermal imaging for respiratory rate analysis. Within the frame of our study, we analysed the aptness of the methods, comparing load assessment for low and high cognitive load tasks under both stationary and mobile conditions. Using an XGBoost classifier, our model achieved 86.11% accuracy for binary cognitive load assessment (low vs. high cognitive load) and 84.24% accuracy for four-way classification (cognitive load$\times$mobility). Feature importance analysis revealed that robust predictors included gaze dynamics (e.g., fixation, pursuit, and saccade durations), pupil diameter metrics (such as FFT band power and variability measures), and facial and respiratory features (including brow lowering and nostril temperature quantiles) for assessing cognitive load in mobile AR.

Source confusion occurs when individuals attribute a memory to the wrong source (e.g., confusing a picture with an experienced event). Virtual Reality (VR) represents a new source of memories particularly prone to being confused with reality. While previous research identified causes of source confusion between reality and other sources (e.g., imagination, pictures), there is currently no understanding of what characteristics specific to VR (e.g., immersion, presence) could influence source confusion. Through a laboratory study (n=29), we 1) confirm the existence of VR source confusion with current technology, and 2) present a quantitative and qualitative exploration of factors influencing VR source confusion. Building on the Source Monitoring Framework, we identify VR characteristics and assumptions about VR capabilities (e.g., poor rendering) that are used to distinguish virtual from real memories. From these insights, we reflect on how the increasing realism of VR could leave users vulnerable to memory errors and perceptual manipulations.

Hybrid board games (HBGs) augment their analog origins digitally (e.g., through apps) and are an increasingly popular pastime activity. Continuous world and character development and customization, known to facilitate engagement in video games, remain rare in HBGs. If present, they happen digitally or imaginarily, often leaving physical aspects generic. We developed DungeonMaker, a fabrication-augmented HBG bridging physical and digital game elements: 1) the setup narrates a story and projects a digital game board onto a laser cutter; 2) DungeonMaker assesses player-crafted artifacts; 3) DungeonMaker’s modified laser head senses and moves player- and non-player figures, and 4) can physically damage figures. An evaluation (n = 4 × 3) indicated that DungeonMaker provides an engaging experience, may support players’ connection to their figures, and potentially spark novices’ interest in fabrication. DungeonMaker provides a rich constellation to play HBGs by blending aspects of craft and automation to couple the physical and digital elements of an HBG tightly.

Extended Reality (XR) allows in-situ previewing of designs to be manufactured through Personal Fabrication (PF). These in-situ interactions exhibit advantages for PF, like incorporating the environment into the design process. However, design-for-fabrication in XR often happens through either highly complex 3D-modeling or is reduced to rudimentary adaptations of crowd-sourced models. We present pARam, a tool combining parametric designs (PDs) and XR, enabling in-situ configuration of artifacts for PF. In contrast to modeling- or search-focused approaches, pARam supports customization through embodied and practical inputs (e.g., gestures, recommendations) and evaluation (e.g., lighting estimation) without demanding complex 3D-modeling skills. We implemented pARam for HoloLens 2 and evaluated it (n = 20), comparing XR and desktop conditions. Users succeeded in choosing context-related parameters and took their environment into account for their configuration using pARam. We reflect on the prospects and challenges of PDs in XR to streamline complex design methods for PF while retaining suitable expressivity.

In the near future ubiquitous Augmented Reality (AR) will see virtual spatial content seamlessly integrated into our everyday lives through fashionable, wearable devices such as AR glasses. In doing so, we will unlock the capacity for multiple stakeholders to augment and personalize our view of reality - not always to the benefit of users. Using speculative design, we explore the risks and repercussions of third party-driven AR-enacted manipulation and deception through the lens of supermarket grocery shopping - an activity where consumers are routinely tracked and exposed to manipulation of attention and purchasing decisions. Through a scenario generation activity, 20 participants (mixing both existing XR users and frequent shoppers) co-created 58 scenarios reflecting on how AR-driven manipulation or deception of shoppers could be enacted. Our results show that (1) manipulation, rather than deception, is the primary means of affecting consumer behaviour, necessitating we consider AR Manipulative and Deceptive Designs (MDDs) more broadly. Moreover (2), we highlight a) how known MDDs can manifest in AR, and b) four novel MDDs that are specific to AR: Redirected Navigation, Directed & Forced Attention Shifts, Reality Interference and Delayed and Detained. We reflect on the stakeholders that would manipulate consumer perception of reality, the different manipulations enacted across the lifecycle of consumer shopping, and the AR elements exploited to deliver these ARMDDs, deriving insights into future harms, ethics and safeguarding around ARMDDs to minimize their impact.

Virtual Reality (VR) technologies become ubiquitous, allowing people to employ immersive experiences in their homes. Since VR participants are visually disconnected from their real-world environment, commercial products propose safety boundaries to prevent colliding with their surroundings. However, there is a lack of empirical knowledge on how people perceive and interact with safety boundaries in everyday VR usage. This paper investigates this research gap with two mixed-method empirical studies. Study 1 reports an online survey (n=48) collecting data about attitudes towards safety boundaries, behavior while interacting with them, and reasons for breakout. Our analysis with open coding reveals that some VR participants ignored safety boundaries intentionally, even breaking out of them and continuing their actions. Study 2 investigates how and why VR participants intentionally break out when interacting close to safety boundaries and obstacles by replicating breakouts in a lab study (n=12). Our interview and breakout data discover three strategies, revealing VR participants sometimes break out of boundaries based on their real-world spatial information. Finally, we discuss improving future VR safety mechanisms by supporting participants’ real-world spatial mental models using landmarks.

On-body menus present a novel interaction paradigm within Virtual Reality (VR) environments by embedding virtual interfaces directly onto the user’s body. Unlike traditional screen-based interfaces, on-body menus enable users to interact with virtual options or icons visually attached to their physical form. In this paper, We investigated the impact of the creation process on the effectiveness of on-body menus, comparing first-person, third-person, and mirror perspectives. Our first study ( N = 12) revealed that the mirror perspective led to faster creation times and more accurate recall compared to the other two perspectives. To further explore user preferences, we conducted a second study ( N = 18) utilizing a VR system with integrated body tracking. By combining distributions of icons from both studies ( N = 30), we confirmed significant preferences in on-body menu placement based on icon category ( e.g. , Social Media icons were consistently placed on forearms). We also discovered associations between categories, such as Leisure and Social Media icons frequently co-occurring. Our findings highlight the importance of the creation process, uncover user preferences for on-body menu organization, and provide insights to guide the development of intuitive and effective on-body interactions within virtual environments.

Dark patterns are deceptive designs that influence a user’s interactions with an interface to benefit someone other than the user. Prior work has identified dark patterns in windows, icons, menus, and pointer (WIMP) interfaces and ubicomp environments, but how dark patterns can manifest in Augmented and Virtual Reality (collectively XR) requires more attention. We therefore conducted 10 co-design workshops with 20 experts in XR and deceptive design. Our participants co-designed 42 scenarios containing dark patterns, based on application archetypes presented in recent HCI/XR literature. In the co-designed scenarios, we identified 10 novel dark patterns in addition to 39 existing ones, as well as 10 examples in which specific characteristics associated with XR potentially amplified the effect dark patterns could have on users. Based on our findings and prior work, we present a classification of XR-specific properties that facilitate dark patterns: perception, spatiality, physical/virtual barriers, and XR device sensing. We also present the experts’ assessments of the likelihood and severity of the co-designed scenarios and highlight key aspects they considered for this evaluation, for example, technological feasibility, ease of upscaling and distributing malicious implementations, and the application’s context of use. Finally, we discuss means to mitigate XR dark patterns and support regulatory bodies to reduce potential harms.

Whole-body movements enhance the presence and enjoyment of Virtual Reality (VR) experiences. However, using large gestures is often uncomfortable and impossible in confined spaces (e.g., public transport). We introduce FingerMapper, mapping small-scale finger motions onto virtual arms and hands to enable whole-body virtual movements in VR. In a first target selection study (n=13) comparing FingerMapper to hand tracking and ray-casting, we found that FingerMapper can significantly reduce physical motions and fatigue while having a similar degree of precision. In a consecutive study (n=13), we compared FingerMapper to hand tracking inside a confined space (the front passenger seat of a car). The results showed participants had significantly higher perceived safety and fewer collisions with FingerMapper while preserving a similar degree of presence and enjoyment as hand tracking. Finally, we present three example applications demonstrating how FingerMapper could be applied for locomotion and interaction for VR in confined spaces.

Human memory has notable limitations (e.g., forgetting) which have necessitated a variety of memory aids (e.g., calendars). As we grow closer to mass adoption of everyday Extended Reality (XR), which is frequently leveraging perceptual limitations (e.g., redirected walking), it becomes pertinent to consider how XR could leverage memory limitations (forgetting, distorting, persistence) to induce memory manipulations. As memories highly impact our self-perception, social interactions, and behaviors, there is a pressing need to understand XR Memory Manipulations (XRMMs). We ran three speculative design workshops (n=12), with XR and memory researchers creating 48 XRMM scenarios. Through thematic analysis, we define XRMMs, present a framework of their core components and reveal three classes (at encoding, pre-retrieval, at retrieval). Each class differs in terms of technology (AR, VR) and impact on memory (influencing quality of memories, inducing forgetting, distorting memories). We raise ethical concerns and discuss opportunities of perceptual and memory manipulations in XR.

Everyday Augmented Reality (AR) displays, with wearable, fashionable, all-day form factors, may one day supplant our reliance on physical displays, heralding new capabilities in augmented intelligence and perception, communication, productivity, and more. Such technology has the potential to become as fundamental to our daily lives as smartphones are today, empowering users, communities, business, governments, and others to alter, augment, diminish or otherwise mediate our perception of reality. For social good, this technology can enable augmenting expression of social identity to better represent our ‘authentic’ self, and virtually enhancing real-world social spaces to encourage greater community ownership and social cohesion. For social harm however, everyday AR could facilitate and amplify manipulation, information disorder (e.g. dis-information), censorship and coercion in our day-to-day experience of reality. In this essay, we consider some of the key societal changes (and ethical challenges) posed by the adoption of everyday AR, and argue that everyday AR will provoke the need for new human rights to be considered alongside proposed neurorights and existing and envisaged digital human rights, around: who can mediate reality (perceptual autonomy); what elements of reality are permissible to alter/augment (perceptual agency); and governing permissible intent regarding why we augment the user’s perception of reality, in particular considering tensions in cognitive autonomy (e.g. manipulation) and perceptual integrity (e.g. information disorder).

Augmented Reality (AR) can alter environments and steer attention. While prior work dominantly focuses on exploring performances of augmentations, this work aims to understand the societal impact of AR in complex social situations. Focusing on prosocial helping behaviour, we created two scenarios and designed five augmentations aiming to motivate a user to help. We wanted to understand (1) the impact on situation perception and (2) the impact on the social structure. In an online video experiment (n=294), we found that augmenting can impact anxiety about the situation and significantly increase the perceived reason to help being directed towards the augmentation. Similarly, we found that the helped rated the "reason" and "thankfulness" significantly higher towards AR than the helper, creating a disagreement around agency and responsibility. We discuss the implications of AR in complex social structures and how responsibility and agency will become important when embedding AR in our social lives.

Today's social media (SM) platforms are toolkits consisting of features with different use cases, some strongly related to habitual and regretful use. Especially Infinite Scrolling (IS) has been reported to make users feel like they are being caught in a loop, regretfully elongating SM sessions. We investigated and defined this loop while unveiling the processes that make users break it. Based on a one-week-long field study (N=46), we unfolded and categorized general reasons for leaving social media and related those to IS. In light of our findings, we argue that SM interventions should not only focus on the app but incorporate the user's context, as most reasons to break SM sessions were not related to the app but the user's general context. Our findings and prior work also indicate the coexistence of multiple loops, which we define as inner (intra-session) loops surrounded by an outer (habitual) loop.

3D-printers enable end-users to design and fabricate unique physical artifacts but maintain an increased entry barrier and friction. End users must design tangible artifacts through intangible media away from the main problem space (ex-situ) and transfer spatial requirements to an abstract software environment. To allow users to evaluate dimensions, balance, or fit early and in-situ, we developed BrickStARt, a design tool using tangible construction blocks paired with a mixed-reality headset. Users assemble a physical block model at the envisioned location of the fabricated artifact. Designs can be tested tangibly, refined, and digitally post-processed, remaining continuously in-situ. We implemented BrickStARt using a Magic Leap headset and present walkthroughs, highlighting novel interactions for 3D-design. In a user study (n=16), first-time 3D-modelers succeeded more often using BrickStARt than Tinkercad. Our results suggest that BrickStARt provides an accessible and explorative process while facilitating quick, tangible design iterations that allow users to detect physics-related issues (e.g., clearance) early on.

“Virtual-Physical Perceptual Manipulations” (VPPMs) such as redirected walking and haptics expand the user’s capacity to interact with Virtual Reality (VR) beyond what would ordinarily physically be possible. VPPMs leverage knowledge of the limits of human perception to effect changes in the user’s physical movements, becoming able to (perceptibly and imperceptibly) nudge their physical actions to enhance interactivity in VR. We explore the risks posed by the malicious use of VPPMs. First, we define, conceptualize and demonstrate the existence of VPPMs. Next, using speculative design workshops, we explore and characterize the threats/risks posed, proposing mitigations and preventative recommendations against the malicious use of VPPMs. Finally, we implement two sample applications to demonstrate how existing VPPMs could be trivially subverted to create the potential for physical harm. This paper aims to raise awareness that the current way we apply and publish VPPMs can lead to malicious exploits of our perceptual vulnerabilities.

Diminished reality (DR) refers to the concept of removing content from a user’s visual environment. While its implementation is becoming feasible, it is still unclear how users perceive and interact in DR-enabled environments and what applications it benefits. To address this challenge, we first conduct a formative study to compare user perceptions of DR and mediated reality effects (e. g., changing the color or size of target elements) in four example scenarios. Participants preferred removing objects through opacity reduction (i. e., the standard DR implementation) and appreciated mechanisms for maintaining a contextual understanding of diminished items (e. g., outlining). In a second study, we explore the user experience of performing tasks within DR-enabled environments. Participants selected which objects to diminish and the magnitude of the effects when performing two separate tasks (video viewing, assembly). Participants were comfortable with decreased contextual understanding, particularly for less mobile tasks. Based on the results, we define guidelines for creating general DR-enabled environments.

Cross-reality systems empower users to transition along the reality-virtuality continuum or collaborate with others experiencing different manifestations of it. However, prototyping these systems is challenging, as it requires sophisticated technical skills, time, and often expensive hardware. We present VRception, a concept and toolkit for quick and easy prototyping of cross-reality systems. By simulating all levels of the reality-virtuality continuum entirely in Virtual Reality, our concept overcomes the asynchronicity of realities, eliminating technical obstacles. Our VRception Toolkit leverages this concept to allow rapid prototyping of cross-reality systems and easy remixing of elements from all continuum levels. We replicated six cross-reality papers using our toolkit and presented them to their authors. Interviews with them revealed that our toolkit sufficiently replicates their core functionalities and allows quick iterations. Additionally, remote participants used our toolkit in pairs to collaboratively implement prototypes in about eight minutes that they would have otherwise expected to take days.

Social Media (SM) has shown that we adapt our communication and disclosure behaviors to available technological opportunities. Head-mounted Augmented Reality (AR) will soon allow to effortlessly display the information we disclosed not isolated from our physical presence (e.g., on a smartphone) but visually attached to the human body. In this work, we explore how the medium (AR vs. Smartphone), our role (being augmented vs. augmenting), and characteristics of information types (e.g., level of intimacy, self-disclosed vs. non-self-disclosed) impact the users’ comfort when displaying personal information. Conducting an online survey (N=148), we found that AR technology and being augmented negatively impacted this comfort. Additionally, we report that AR mitigated the effects of information characteristics compared to those they had on smartphones. In light of our results, we discuss that information augmentation should be built on consent and openness, focusing more on the comfort of the augmented rather than the technological possibilities.

Personal fabrication is made more accessible through repositories like Thingiverse, as they replace modeling with retrieval. However, they require users to translate spatial requirements to keywords, which paints an incomplete picture of physical artifacts: proportions or morphology are non-trivially encoded through text only. We explore a vision of in-situ spatial search for (future) physical artifacts, and present ShapeFindAR, a mixed-reality tool to search for 3D models using in-situ sketches blended with textual queries. With ShapeFindAR, users search for geometry, and not necessarily precise labels, while coupling the search process to the physical environment (e.g., by sketching in-situ, extracting search terms from objects present, or tracing them). We developed ShapeFindAR for HoloLens 2, connected to a database of 3D-printable artifacts. We specify in-situ spatial search, describe its advantages, and present walkthroughs using ShapeFindAR, which highlight novel ways for users to articulate their wishes, without requiring complex modeling tools or profound domain knowledge.

Personal fabrication empowers users to create objects increasingly easier and faster. This continuous decrease in effort evokes a speculative scenario of Ephemeral Fabrication (EF), enabled and amplified by emerging paradigms of mobile, wearable, or even body-integrated fabrication. EF yields fast, temporary, in-situ solutions for everyday problems (e.g., creating a protective skin, affixing a phone). Users solely create those, since the required effort is negligible. We present and critically reflect on the EF scenario, by exploring current trends in research and building a body-worn fabrication device. EF is a plausible extrapolation of current developments, entailing both positive (e.g., accessibility) and negative implications (e.g., unsustainability). Using speculative design methodology to question the trajectory of personal fabrication, we argue that to avert the aftermath of such futures, topics like sustainability can not remain an afterthought, but rather be situated in interactions themselves: through embedded constraints, conscious material choice, and constructive embedding of ephemerality.

Digital eye strain (DES), caused by prolonged exposure to digital screens, stresses the visual system and negatively affects users’ well-being and productivity. While DES is well-studied in computer displays, its impact on users of virtual reality (VR) head-mounted displays (HMDs) is largely unexplored—despite that some of their key properties (e.g., the vergence-accommodation conflict) make VR-HMDs particularly prone. This work provides the first comprehensive investigation into DES in VR HMDs. We present results from a survey with 68 experienced users to understand DES symptoms in VR-HMDs. To help address DES, we investigate eye exercises resulting from survey answers and blue light filtering in three user studies (N = 71). Results demonstrate that eye exercises, but not blue light filtering, can effectively reduce DES. We conclude with an extensive analysis of the user studies and condense our findings in 10 key challenges that guide future work in this emerging research area.

Based on a systematic literature review of more than 300 papers published over the last 10 years, we provide indicators that the simulator sickness questionnaire (SSQ) is extensively used and widely accepted as a general discomfort measure in virtual reality (VR) research – although it actually only accounts for one category of symptoms. This results in important other categories (digital eye strain (DES) and ergonomics) being largely neglected. To contribute to a more comprehensive picture of discomfort in VR head-mounted displays, we further conducted an online study (N=352) on the severity and relevance of all three symptom categories. Most importantly, our results reveal that symptoms of simulator sickness are significantly less severe and of lower prevalence than those of DES and ergonomics. In light of these findings, we critically discuss the current use of SSQ as the only discomfort measure and propose a more comprehensive factor model that also includes DES and ergonomics.

Spatial computing devices that blend virtual and real worlds have the potential to soon become ubiquitous. Yet, creating experiences for spatial computing is non-trivial and needs skills in programming and 3D content creation, rendering them inaccessible to a wider group of users. We present SpatialProto, an in-situ spatial prototyping system for lowering the barrier to engage in spatial prototyping. With a depth-sensing capable mixed reality headset, SpatialProto lets users record animated objects of the real-world environment (e.g. paper, clay, people, or any other prop), extract only the relevant parts, and directly place and transform these recordings in their physical environment. We describe the design and implementation of SpatialProto, a user study evaluating the system’s prototype with non-expert users (n = 9), and demonstrate applications where multiple captures are fused for compelling augmented reality experiences.

Augmented Reality (AR) glasses equip users with the tools to modify the visual appearance of their surrounding environment. This might severely impact interpersonal communication, as the conversational partners will no longer share the same visual perception of reality. Grounded in color-in-context theory, we present a potential AR application scenario in which users can modify the color of the environment to achieve subconscious benefits. In a consecutive online survey (N=64), we measured the user’s comfort, acceptance of altering and being altered, and how it is impacted by being able to perceive or not perceive the alteration. We identified significant differences depending on (1) who or what is the target of the alteration, (2) which body part is altered, and (3) which relationship the conversational partners share. In light of our quantitative and qualitative findings, we discuss ethical and practical implications for future devices and applications that employ visual alterations.

Automated vehicles will change the trucking industry as human drivers become more absent. In crossing scenarios, external communication concepts are already evaluated to resolve potential issues. However, automated delivery poses unique communication problems. One specific situation is the delivery to the curb with the truck remaining partially on the street, blocking sidewalks. Here, pedestrians have to walk past the vehicle with reduced sight, resulting in safety issues. To address this, we conducted a literature survey revealing the lack of addressing external communication of automated vehicles in situations other than crossings. Afterwards, a study in Virtual Reality (N=20) revealed the potential of such communication. While the visualization (e.g., arrows or text) of whether it is safe to walk past the truck only played a minor part, the information of being able to safely walk past was highly appreciated. This shows that external communication concepts carry great potential besides simple crossing scenarios.

Virtual and augmented reality head-mounted displays (HMDs) are currently heavily relying on spatially tracked input devices (STID) for interaction. These STIDs are all prone to the phenomenon that a discrete input (e.g. button press) will disturb the position of the tracker, resulting in a different selection point during ray-cast interaction (Heisenberg Effect of Spatial Interaction). Besides the knowledge of its existence, there is currently a lack of a deeper understanding of its severity, structure and impact on throughput and angular error during a selection task. In this work, we present a formal evaluation of the Heisenberg effect and the impact of body posture, arm position and STID degrees of freedom on its severity. In a Fitt's Law inspired user study (N=16), we found that the Heisenberg effect is responsible for 30.45% of the overall errors occurring during a pointing task, but can be reduced by 25.4% using a correction function.

People with vision impairments (VIP) are among the most vulnerable road users in traffic. Autonomous vehicles are believed to reduce accidents but still demand some form of external communication signaling relevant information to pedestrians. Recent research on the design of vehicle-pedestrian communication (VPC) focuses strongly on concepts for a non-disabled population. Our work presents an inclusive user-centered design for VPC, beneficial for both vision impaired and seeing pedestrians. We conducted a workshop with VIP (N=6), discussing current issues in road traffic and comparing communication concepts proposed by literature. A thematic analysis unveiled two important themes: number of communicating vehicles and content (affecting duration). Subsequently, we investigated these in a second user study in virtual reality (N=33, 8 VIP) comparing the VPC between groups of abilities. We found that trust and understanding is enhanced and cognitive load reduced when all relevant vehicles communicate; high content messages also reduce cognitive load.

Sketching in virtual reality (VR) enhances perception and understanding of 3D volumes, but is currently a challenging task, as spatial input devices (e.g., tracked controllers) do not provide any scaffolding or constraints for mid-air interaction. We present VRSketchIn, a VR sketching application using a 6DoF-tracked pen and a 6DoF-tracked tablet as input devices, combining unconstrained 3D mid-air with constrained 2D surface-based sketching. To explore what possibilities arise from this combination of 2D (pen on tablet) and 3D input (6DoF pen), we present a set of design dimensions and define the design space for 2D and 3D sketching interaction metaphors in VR. We categorize prior art inside our design space and implemented a subset of metaphors for pen and tablet sketching in our prototype. To gain a deeper understanding which specific sketching operations users perform with 2D and which with 3D metaphors, we present findings of usability walkthroughs with six participants.

The accessibility of tools to model artifacts is one of the core driving factors for the adoption of Personal Fabrication. Subsequently, model repositories like Thingiverse became important tools in (novice) makers' processes. They allow them to shorten or even omit the design process, offloading a majority of the effort to other parties. However, steps like measurement of surrounding constraints (e.g., clearance) which exist only inside the users' environment, can not be similarly outsourced. We propose Mix&Match a mixed-reality-based system which allows users to browse model repositories, preview the models in-situ, and adapt them to their environment in a simple and immediate fashion. Mix&Match aims to provide users with CSG operations which can be based on both virtual and real geometry. We present interaction patterns and scenarios for Mix&Match, arguing for the combination of mixed reality and model repositories. This enables almost modelling-free personal fabrication for both novices and expert makers.

Media (e.g. videos, images, and text) shared on social platforms such as Facebook and WeChat are often visually enriched through digital content (e.g. emojis, stickers, animal faces) increasing joy, personalization, and expressiveness. While voice messages (VMs) are experiencing a high frequent usage, they currently lack any form of digital augmentation. This work is the first to present and explore the concept of augmented VMs. Inspired by visual augmentations we designed and implemented an editor, allowing users to enhance VMs with background sounds, voice changers, and sound stickers. In a first evaluation (N = 15) we found that participants used augmentations frequently (2.73 per message on average) and rated augmented VMs to be expressive, personal and more fun than ordinary VMs. In a consecutive step, we analyzed the 45 augmented VMs recorded during the study and identified three distinct message types (decoration, composition and integrated) that inform about potential usage.

Adoption and use of smartphone-based asynchronous voice messaging has increased substantially in recent years. However, this communication channel has a strong tendency to polarize. To provide an understanding of this modality, we started by conducting an online survey (n=1,003) exploring who is using voice messages, their motives, and utilization. In a consecutive field study (n=6), we analyzed voice messaging behavior of six avid voice message users in a two-week field study, followed by semi-structured interviews further exploring themes uncovered in our survey. Conducting a thematic analysis, we identified four themes driving voice messaging usage: convenience, para-linguistic features, situational constraints and the receiver. Voice messaging helps to overcome issues of mobile communication, through ease of use, asynchronous implementation, and voices’ rich emotional context. It also was perceived as enabling more efficient communication, helps to handle secondary occupations, and better facilitates maintenance of close relationships. Despite the increased effort required to listen to a voice message, they complement communication with people we care about.

Head-Mounted Displays (HMDs) are the dominant form of enabling Virtual Reality (VR) and Augmented Reality (AR) for personal use. One of the biggest challenges of HMDs is the exclusion of people in the vicinity, such as friends or family. While recent research on asymmetric interaction for VR HMDs has contributed to solving this problem in the VR domain, AR HMDs come with similar but also different problems, such as conflicting information in visualization through the HMD and projection. In this work, we propose ShARe, a modified AR HMD combined with a projector that can display augmented content onto planar surfaces to include the outside users (non-HMD users). To combat the challenge of conflicting visualization between augmented and projected content, ShARe visually aligns the content presented through the AR HMD with the projected content using an internal calibration procedure and a servo motor. Using marker tracking, non-HMD users are able to interact with the projected content using touch and gestures. To further explore the arising design space, we implemented three types of applications (collaborative game, competitive game, and external visualization). ShARe is a proof-of-concept system that showcases how AR HMDs can facilitate interaction with outside users to combat exclusion and instead foster rich, enjoyable social interactions.

Current solutions for creating co-located Mixed Reality (MR) experiences typically rely on platform-specific synchronisation of spatial anchors or Simultaneous Localisation and Mapping (SLAM) data across clients, often coupled to cloud services. This introduces significant costs (in development and deployment), constraints (with interoperability across platforms often limited), and privacy concerns. For practitioners, support is needed for creating platform-agnostic co-located MR experiences. This paper explores the utility of aligned SLAM solutions by 1) surveying approaches toward aligning disparate device coordinate spaces, formalizing their theoretical accuracy and limitations; 2) providing skeleton implementations for audience-based, small-scale and large-scale co-location using said alignment approaches; and 3) detailing how we can assess the accuracy and safety of 6DoF/SLAM tracking solutions for any arbitrary device and dynamic environment without the need for an expensive ground truth optical tracking, by using trilateration and a $30 laser distance meter. Through this, we hope to further democratise the creation of cross-platform co-located MR experiences.

Augmented and virtual reality (AR/VR) has entered the mass market and, with it, will soon eye tracking as a core technology for next generation head-mounted displays (HMDs). In contrast to existing gaze interfaces, the 3D nature of AR and VR requires estimating a user's gaze in 3D. While first applications, such as foveated rendering, hint at the compelling potential of combining HMDs and gaze, a systematic analysis is missing. To fill this gap, we present the first design space for gaze interaction on HMDs. Our design space covers human depth perception and technical requirements in two dimensions aiming to identify challenges and opportunities for interaction design. As such, our design space provides a comprehensive overview and serves as an important guideline for researchers and practitioners working on gaze interaction on HMDs. We further demonstrate how our design space is used in practice by presenting two interactive applications: EyeHealth and XRay-Vision.

Current approaches for locomotion in virtual reality are either creating a visual-vestibular conflict, which is assumed to cause simulator sickness, or use metaphors such as teleportation to travel longer distances, lacking the perception of self motion. We propose VRSpinning, a seated locomotion approach based around stimulating the user's vestibular system using a rotational impulse to induce the perception of linear self-motion. In a first study we explored the approach of oscillating the chair in different frequencies during visual forward motion and collected user preferences on applying these feedback types. In a second user study we used short bursts of rotational acceleration to match the visual forward acceleration. We found that this rotational stimulus significantly reduced simulator sickness and increased the perception of self-motion in comparison to no physical motion.

Mobile VR HMDs enable scenarios where they are being used in public, excluding all the people in the surrounding (Non-HMD Users) and reducing them to be sole bystanders. We present FaceDisplay, a modified VR HMD consisting of three touch sensitive displays and a depth camera attached to its back. People in the surrounding can perceive the virtual world through the displays and interact with the HMD user via touch or gestures. To further explore the design space of FaceDisplay, we implemented three applications (FruitSlicer, SpaceFace and Conductor) each presenting different sets of aspects of the asymmetric co-located interaction (e.g. gestures vs touch). We conducted an exploratory user study (n=16), observing pairs of people experiencing two of the applications and showing a high level of enjoyment and social interaction with and without an HMD. Based on the findings we derive design considerations for asymmetric co-located VR applications and argue that VR HMDs are currently designed having only the HMD user in mind but should also include Non-HMD Users.

Virtual reality (VR) technology strives to enable a highly immersive experience for the user by including a wide variety of modalities (e.g. visuals, haptics). Current VR hardware however lacks a sufficient way of communicating the perception of weight of an object, resulting in scenarios where users can not distinguish between lifting a bowling ball or a feather. We propose a solely software based approach of simulating weight in VR by deliberately using perceivable tracking offsets. These tracking offsets nudge users to lift their arm higher and result in a visual and haptic perception of weight. We conducted two user studies showing that participants intuitively associated them with the sensation of weight and accept them as part of the virtual world. We further show that compared to no weight simulation, our approach led to significantly higher levels of presence, immersion and enjoyment. Finally, we report perceptional thresholds and offset boundaries as design guidelines for practitioners.

Redirected walking (RDW) allows virtual reality (VR) users to walk infinitely while staying inside a finite physical space through subtle shifts (gains) of the scene to redirect them back inside the volume. All prior approaches measure the feasibility of RDW techniques based on if the user perceives the manipulation, leading to rather small applicable gains. However, we treat RDW as an interaction technique and therefore use visually perceivable gains instead of using the perception of manipulation. We revisited prior experiments with focus on applied gains and additionally tested higher gains on the basis of applicability in a user study. We found that users accept curvature gains up to 20°/m, which reduces the necessary physical volume down to approximately 6x6m for virtually walking infinitely straight ahead. Our findings strife to rethink the usage of redirection from being unperceived to being applicable and natural.

Most current virtual reality (VR) head-mounted displays (HMD) create a highly immersive experience and are currently becoming part of the living room entertainment (e.g. PSVR). However, current VR systems focus mainly on increasing the immersion and enjoyment for the user wearing the HMD (HMD user). This results in all the bystanders (Non-HMD users) in the living room being excluded from the experience and degraded to mainly observing the HMD user. In this demonstration we show ShareVR, a VR system using floor projection and mobile displays in combination with positional tracking to visualize the virtual world for the Non-HMD user, enabling them to interact with the HMD user and become part of the VR experience. Additionally, we implemented several experiences for the asymmetric nature of ShareVR, exploring its design space.

Mobile VR HMDs enable users to experience virtual reality content in a variety of nomadic scenarios, excluding all the people in the surrounding (Non-HMD Users) and reducing them to be sole bystanders. This leads to a scenario where the HMD User experiences a sense of isolation and the Non-HMD Users a sense of exclusion. To battle these phenomena we present FaceDisplay, a modified VR HMD consisting of three touch sensitive displays and a depth camera attached to its back. This allows Non-HMD User to see inside the immersed users virtual world and enable them to interact via touch and gestures. We built a VR HMD prototype consisting of three additional screens and present interaction techniques and an example application that leverage the FaceDisplay design space.

Virtual reality (VR) head-mounted displays (HMD) allow for a highly immersive experience and are currently becoming part of the living room entertainment. Current VR systems focus mainly on increasing the immersion and enjoyment for the user wearing the HMD (HMD user), resulting in all the bystanders (Non-HMD users) being excluded from the experience. We propose ShareVR, a proof-of-concept prototype using floor projection and mobile displays in combination with positional tracking to visualize the virtual world for the Non-HMD user, enabling them to interact with the HMD user and become part of the VR experience. We designed and implemented ShareVR based on the insights of an initial online survey (n=48) with early adopters of VR HMDs. We ran a user study (n=16) comparing ShareVRto a baseline condition showing how the interaction using ShareVR led to an increase of enjoyment, presence and social interaction. In a last step we implemented several experiences for ShareVR, exploring its design space and giving insights for designers of co-located asymmetric VR experiences.

Mobile virtual reality (VR) head-mounted displays (HMDs) allow users to experience highly immersive entertainment whilst being in a mobile scenario. Long commute times make casual gaming in public transports and cars a common occupation. However, VR HMDs can currently not be used in moving vehicles since the car's rotation affects the HMD's sensors and simulator sickness occurs when the visual and vestibular system are stimulated with incongruent information. We present CarVR, a solution to enable VR in moving vehicles by subtracting the car's rotation and mapping vehicular movements with the visual information. This allows the user to actually feel correct kinesthetic forces during the VR experience. In a user study (n = 21), we compared CarVR inside a moving vehicle with the baseline of using VR without vehicle movements. We show that the perceived kinesthetic forces caused by CarVR increase enjoyment and immersion significantly while simulator sickness is reduced compared to a stationary VR experience. Finally, we explore the design space of in-car VR entertainment applications using real kinesthetic forces and derive design considerations for practitioners.

Deaf individuals often experience communication difficulties in face-to-face interactions with hearing people. In order to support deaf individuals in such situations, an active stream of assistive technology (AT) research focuses on real-time translation of sign language. We investigate the impact of real-time translation-based ATs on communication quality between deaf and hearing individuals. We conducted a focus group and aWizard of Oz study in which deaf and hearing participants jointly interacted with different assistive technologies. We find that while ATs facilitate communication, communication quality is degraded by to breaks in the conversation. Using Co-Cultural Theory, we identify deaf people as a subordinate group inside a hearing society. Our results indicate that current ATs reinforce this subordination by emphasizing deficiency of mastering the dominant form of communication. Based on our findings, we propose a change in design perspective by enabling the hearing to sign rather than the deaf to "hear". We argue that ATs should not be seen as "just" a tool for the Deaf but rather as a collaborative technology.

We present SwiVRChair, a motorized swivel chair to nudge users' orientation in 360 degree storytelling scenarios. Since rotating a scene in virtual reality (VR) leads to simulator sickness, storytellers currently have no way of controlling users' attention. SwiVRChair allows creators of 360 degree VR movie content to be able to rotate or block users' movement to either show certain content or prevent users from seeing something. To enable this functionality, we modified a regular swivel chair using a 24V DC motor and an electromagnetic clutch. We developed two demo scenarios using both mechanisms (rotate and block) for the Samsung GearVR and conducted a user study (n=16) evaluating the presence, enjoyment and simulator sickness for participants using SwiVRChair compared to self control (Foot Control). Users rated the experience using SwiVRChair to be significantly more immersive and enjoyable whilst having a decrease in simulator sickness.

We present FaceTouch, a novel interaction concept for mobile Virtual Reality (VR) head-mounted displays (HMDs) that leverages the backside as a touch-sensitive surface. With FaceTouch, the user can point at and select virtual content inside their field-of-view by touching the corresponding location at the backside of the HMD utilizing their sense of proprioception. This allows for rich interaction (e.g. gestures) in mobile and nomadic scenarios without having to carry additional accessories (e.g. a gamepad). We built a prototype of FaceTouch and conducted two user studies. In the first study we measured the precision of FaceTouch in a display-fixed target selection task using three different selection techniques showing a low error rate of 2% indicate the viability for everyday usage. To asses the impact of different mounting positions on the user performance we conducted a second study. We compared three mounting positions of the touchpad (face, hand and side) showing that mounting the touchpad at the back of the HMD resulted in a significantly lower error rate, lower selection time and higher usability. Finally, we present interaction techniques and three example applications that explore the FaceTouch design space.

We present GyroVR, head worn flywheels designed to render inertia in Virtual Reality (VR. Motions such as flying, diving or floating in outer space generate kinesthetic forces onto our body which impede movement and are currently not represented in VR. We simulate those kinesthetic forces by attaching flywheels to the users head, leveraging the gyroscopic effect of resistance when changing the spinning axis of rotation. GyroVR is an ungrounded, wireless and self contained device allowing the user to freely move inside the virtual environment. The generic shape allows to attach it to different positions on the users body. We evaluated the impact of GyroVR onto different mounting positions on the head (back and front) in terms of immersion, enjoyment and simulator sickness. Our results show, that attaching GyroVR onto the users head (front of the Head Mounted Display (HMD)) resulted in the highest level of immersion and enjoyment and therefore can be built into future VR HMDs, enabling kinesthetic forces in VR.

This paper presents Glass Unlock, a novel concept using smart glasses for smartphone unlocking, which is theoretically secure against smudge attacks, shoulder-surfing, and camera attacks. By introducing an additional temporary secret like the layout of digits that is only shown on the private near-eye display, attackers cannot make sense of the observed input on the almost empty phone screen. We report a user study with three alternative input methods and compare them to current state-of-the-art systems. Our findings show that Glass Unlock only moderately increases authentication times and that users favor the input method yielding the slowest input times as it avoids focus switches between displays.

In this paper we present ColorSnakes, a PIN-based authentication mechanism for smartphones which uses fake paths on a grid of numbers to disguise user input. In a lab study (n=24),we evaluated variations of ColorSnakes in terms of usability and security. In comparison to direct input, indirect input significantly reduced the risk of shoulder surfing (10.5%) without increasing the input time. In a follow up real-world study (n=12), we compared ColorSnakes with PIN entry and Android's Pattern Unlock over the course of three weeks. Although authentication time for ColorSnakes was higher than for the other two mechanisms, participants valued the security benefit over its slightly higher error rate and increased authentication time. We argue that ColorSnakes could be used as an additional authentication mechanism alongside current mechanisms, thus providing the user with the choice of changing to ColorSnakes for certain applications or when there is an observer.

In recent years, the automobile has become more connected and has started offering drivers a variety of services and applications for in-car use, such as Facebook, Twitter, and mobile payment. Many such services require user authentication. We have investigated the requirements and potential of authenticating users with a rotary control knob, which constitutes the central control entity (CCE) in many modern cars. In a user-centered design approach we adapted three established authentication schemes for in-car usage with consideration to driver distraction. We performed extensive user experiments in a driving simulator focusing on usability, security and driver distraction. The results of our experiments provide relevant insights for integrating user authentication methods into the automotive context solely based on present hardware components.