With automated vehicles (AVs), the absence of a human operator could necessitate external Human-Machine Interfaces (eHMIs) to communicate with other road users. Existing research primarily focuses on pedestrian-AV interactions, with limited attention given to other road users, such as cyclists and drivers of manually driven vehicles. So far, no studies have compared the effects of eHMIs across these three road user roles. Therefore, we conducted a within-subjects virtual reality experiment (N=40), evaluating the subjective and objective impact of an eHMI communicating the AV’s intention to pedestrians, cyclists, and drivers under various levels of distraction (no distraction, visual noise, interference). eHMIs positively influenced safety perceptions, trust, perceived usefulness, and mental demand across all roles. While distraction and road user roles showed significant main effects, interaction effects were only observed in perceived usability. Thus, a unified eHMI design is effective, facilitating the standardization and broader adoption of eHMIs in diverse traffic.

Virtual reality exposure therapy (VRET) enables controlled exposure to trauma-related stimuli to facilitate memory access and emotional processing. However, the field remains underexplored for complex post-traumatic stress disorder (C-PTSD). Unlike single-trauma PTSD, C-PTSD requires highly individualized triggers that are difficult to identify and implement safely. We conducted a feasibility study with 11 patients, two trauma therapists, and a VR developer to explore integrating VRET into C-PTSD treatment while safeguarding all stakeholders. Initial findings indicate that simple objects can be just as effective as complex scenes, therapeutic success does not correlate with VR presence levels, and the design process itself became integral to therapy rather than preparatory. However, involving developers in therapy sessions led to considerable emotional stress and role confusion, which required a cautious approach. Based on these insights, we provide methodological recommendations for safe and patient-centered VRET studies that balance therapeutic effectiveness with stakeholder safety across the research process.

Traffic is inherently dangerous, with around 1.19 million fatalities annually. Automotive Mediated Reality (AMR) can enhance driving safety by overlaying critical information (e.g., outlines, icons, text) on key objects to improve awareness, altering objects’ appearance to simplify traffic situations, and diminishing their appearance to minimize distractions. However, real-world AMR evaluation remains limited due to technical challenges. To fill this sim-to-real gap, we present MIRAGE, an open-source tool that enables real-time AMR in real vehicles. MIRAGE implements 15 effects across the AMR spectrum of augmented, diminished, and modified reality using state-of-the-art computational models for object detection and segmentation, depth estimation, and inpainting. In an on-road expert user study (N=9) of MIRAGE, participants enjoyed the AMR experience while pointing out technical limitations and identifying use cases for AMR. We discuss these results in relation to prior work and outline implications for AMR ethics and interaction design.

Eco-friendly service options (EFSOs) aim to reduce personal carbon emissions, yet their eco-friendly framing may permit increased consumption, weakening their intended impact. Such rebound effects remain underexamined in HCI, including how common eco-feedback approaches shape them. We investigate this in an online within-subjects experiment (N = 75) in a ride-hailing context. Participants completed 10 trials for five conditions (No EFSO, EFSO - Minimal, EFSO - CO2 Equivalency, EFSO - Gamified, EFSO - Social), yielding 50 choices between walking and ride-hailing for trips ranging from 0.5mi–2.0mi (≈ 0.80km-3.22km). We measured how different EFSO variants affected ride-hailing uptake relative to a No EFSO baseline. EFSOs lacking explicit eco-feedback metrics increased ride-hailing uptake, and qualitative responses indicate that EFSOs can make convenience-driven choices more permissible. We conclude with implications for designing EFSOs that begin to take rebound effects into account.

Device orchestration in ubiquitous computing remains cumbersome and highly static, particularly given the growing number of personal devices. Routine tasks, such as preparing to read at home, often require multiple manual interactions (lighting, heating, music), which are difficult to automate with rule-based systems due to contextual sensitivity. We present GenX, a perception-driven approach that applies multimodal generative AI to create dynamic, adaptive user experiences. By leveraging vision-language models, GenX interprets a user’s perception of their environment and generates a multimodal UX tailored to available output modalities. The system produces and executes source code in real time, enabling devices to be orchestrated flexibly and contextually. We introduce the concept of Generative Experiences as a scalable solution for dynamic UX design, capable of handling unexpected situations. GenX is demonstrated in two in-the-wild scenarios and a user study, showing positive user perceptions and effectiveness in supporting everyday tasks.

Sensor-based methods such as eye tracking are promising for user modeling in adaptive educational systems, as they enable continuous, fine-grained assessment of learners’ learning processes. However, implementing gaze-based user models requires selecting from a large set of available gaze measures, posing a challenge for adaptive-system designers. To inform this design process, we conduct an exploratory analysis of the associations between gaze measures, learner engagement, and learning outcomes in a video-based learning context. An analysis of data collected from 18 learners indicates that fixation dispersion and fixation rate on active areas of interest are indicators of engagement, while pupil diameter is an indicator of learning outcomes. To support further research on gaze-based user modeling and adaptation, we introduce EnGazement, a dataset comprising over 1.3 million processed eye-tracking data points, enriched with subjective and objective measures of engagement and learning.

The absence of a human operator in automated vehicles (AVs) may require external Human-Machine Interfaces (eHMIs) to facilitate communication with other road users in uncertain scenarios, for example, regarding the right of way. Given the plethora of adjustable parameters, balancing visual and auditory elements is crucial for effective communication with other road users. With N=37 participants, this study employed multi-objective Bayesian optimization to enhance eHMI designs and improve trust, safety perception, and mental demand. By reporting the Pareto front, we identify optimal design trade-offs. This research contributes to the ongoing standardization efforts of eHMIs, supporting broader adoption.

As vehicle automation technology continues to mature, there is a necessity for robust remote monitoring and intervention features. These are essential for intervening during vehicle malfunctions, challenging road conditions, or in areas that are difficult to navigate. This evolution in the role of the human operator - from a constant driver to an intermittent teleoperator - necessitates the development of suitable interaction interfaces. While some interfaces were suggested, a comparative study is missing. We designed, implemented, and evaluated three interaction concepts (path planning, trajectory guidance, and waypoint guidance) with up to four concurrent requests of automated vehicles in a within-subjects study with N=23 participants. The results showed a clear preference for the path planning concept. It also led to the highest usability but lower satisfaction. With trajectory guidance, the fewest requests were resolved. The study's findings contribute to the ongoing development of HMIs focused on the remote assistance of automated vehicles.

Presence is an important and widely used metric to measure the quality of virtual reality (VR) applications. Given the multifaceted and subjective nature of presence, the most common measures for presence are questionnaires. But there is little research on their validity regarding specific presence dimensions and their responsiveness to differences in perception among users. We investigated four presence questionnaires (SUS, PQ, IPQ, Bouchard) on their responsiveness to intensity variations of known presence dimensions and asked users about their consistency with their experience. Therefore, we created five VR scenarios that were designed to emphasize a specific presence dimension. Our findings showed heterogeneous sensitivity of the questionnaires dependent on the different dimensions of presence. This highlights a context-specific suitability of presence questionnaires. The questionnaires' sensitivity was further stated as lower than actually perceived. Based on our findings, we offer guidance on selecting these questionnaires based on their suitability for particular use cases.

Automated vehicle (AV) acceptance relies on their understanding via feedback. While visualizations aim to enhance user understanding of AV's detection, prediction, and planning functionalities, establishing an optimal design is challenging. Traditional "one-size-fits-all" designs might be unsuitable, stemming from resource-intensive empirical evaluations. This paper introduces OptiCarVis, a set of Human-in-the-Loop (HITL) approaches using Multi-Objective Bayesian Optimization (MOBO) to optimize AV feedback visualizations. We compare conditions using eight expert and user-customized designs for a Warm-Start HITL MOBO. An online study (N=117) demonstrates OptiCarVis's efficacy in significantly improving trust, acceptance, perceived safety, and predictability without increasing cognitive load. OptiCarVis facilitates a comprehensive design space exploration, enhancing in-vehicle interfaces for optimal passenger experiences and broader applicability.

Infinite scrolling on social media platforms is designed to encourage prolonged engagement, leading users to spend more time than desired, which can provoke negative emotions. Interventions to mitigate infinite scrolling have shown initial success, yet users become desensitized due to the lack of contextual relevance. Understanding how contextual factors influence intervention effectiveness remains underexplored. We conducted a 7-day user study (N=72) investigating how these contextual factors affect users' reactance and responsiveness to interventions during infinite scrolling. Our study revealed an interplay, with contextual factors such as being at home, sleepiness, and valence playing significant roles in the intervention's effectiveness. Low valence coupled with being at home slows down the responsiveness to interventions, and sleepiness lowers reactance towards interventions, increasing user acceptance of the intervention. Overall, our work contributes to a deeper understanding of user responses toward interventions and paves the way for developing more effective interventions during infinite scrolling.

Automated Urban Air Mobility (UAM) can improve passenger transportation and reduce congestion, but its success depends on passenger trust. While initial research addresses passengers' information needs, questions remain about how to simulate air taxi flights and how these simulations impact users and interface requirements. We conducted a between-subjects study (N=40), examining the influence of motion fidelity in Virtual-Reality-simulated air taxi flights on user effects and interface design. Our study compared simulations with and without motion cues using a 3-Degrees-of-Freedom motion chair. Optimizing the interface design across six objectives, such as trust and mental demand, we used multi-objective Bayesian optimization to determine the most effective design trade-offs. Our results indicate that motion fidelity decreases users' trust, understanding, and acceptance, highlighting the need to consider motion fidelity in future UAM studies to approach realism. However, minimal evidence was found for differences or equality in the optimized interface designs, suggesting personalized interface designs.

The introduction of Highly Automated Vehicles (HAVs) has the potential to increase the independence of blind and visually impaired people (BVIPs). However, ensuring safety and situation awareness when exiting these vehicles in unfamiliar environments remains challenging. To address this, we conducted an interactive workshop with N=5 BVIPs to identify their information needs when exiting an HAV and evaluated three prior-developed low-fidelity prototypes. The insights from this workshop guided the development of PathFinder, a multimodal interface combining visual, auditory, and tactile modalities tailored to BVIP's unique needs. In a three-factorial within-between-subject study with N=16 BVIPs, we evaluated PathFinder against an auditory-only baseline in urban and rural scenarios. PathFinder significantly reduced mental demand and maintained high perceived safety in both scenarios, while the auditory baseline led to lower perceived safety in the urban scenario compared to the rural one. Qualitative feedback further supported PathFinder's effectiveness in providing spatial orientation during exiting.

As the use of Head-Mounted Displays in moving vehicles increases, passengers can immerse themselves in visual experiences independent of their physical environment. However, interaction methods are susceptible to physical motion, leading to input errors and reduced task performance. This work investigates the impact of G-forces, vibrations, and unpredictable maneuvers on 3D interaction methods. We conducted a field study with 24 participants in both stationary and moving vehicles to examine the effects of vehicle motion on four interaction methods: (1) Gaze&Pinch, (2) DirectTouch, (3) Handray, and (4) HeadGaze. Participants performed selections in a Fitts' Law task. Our findings reveal a significant effect of vehicle motion on interaction accuracy and duration across the tested combinations of Interaction Method x Road Type x Curve Type. We found a significant impact of movement on throughput, error rate, and perceived workload. Finally, we propose future research considerations and recommendations on interaction methods during vehicle movement.

Urban Air Mobility (UAM) emerges as a potential solution to urban congestion. However, as it lacks integration with existing transportation systems, methods to study its impact are necessary. Traditional empirical approaches are insufficient to study large-scale effects in this not-yet-real context. We developed UAM-SUMO, an extension of the SUMO simulation platform, to simulate the impact of UAM on public transportation, particularly how air taxis affect traffic flow and mode choices. We detail the modifications to SUMO and the UAM operation parameters. We open-source our code at https://github.com/M-Colley/uam-sumo and present a proof-of-concept data collection and analysis for Ingolstadt, Germany.

Large-scale effects of head-up displays (HUDs) are currently unknown, as experiments focus primarily on empirical data from one participant. Therefore, we simulate the impact of augmented reality (AR) windshield HUDs on driving performance. Using the open-source simulators SUMO and CARLA, we model various AR HUD settings, such as fatigue, and assess their effects on key driving factors such as reaction time, speed adherence, lane-changing behavior, and acceleration. The HUD settings include brightness, information frequency, field of view, and relevance of displayed information. The simulation provides insight into how different HUD configurations may influence driving behavior, contributing to future vehicle design and safety guidelines for AR HUDs.

As living and working spaces become scarce and costly, interiors transitioning between living, working, and sleeping configurations while enabling customized setups are in demand. Traditional furniture consumes space and is cumbersome to rearrange. Shape-changing furniture could solve this, yet existing options lack resolution, stability, or adaptability. We present AirClick, which facilitates on-demand room transformation using modular interactive inflatables. Our fabrication process supports personally fabricated and retrofitted retail inflatables. The touch-actuated modules connect to a floor-based air connector grid, facilitating interaction while integrating with traditional furniture. In a lab study (N=20) across four scenarios (office, meeting room, apartment room, multipurpose hall), participants rapidly transformed rooms and perceived AirClick as significantly more usable with higher intention to use in the everyday scenarios than in the hall, indicating suitability for routine activities with low to medium requirements for robustness. User feedback highlights AirClick ’s usefulness and scalability in diverse settings, hence showing AirClick’s space-saving and customizable design can enhance the functionality and adaptability of living and working spaces.

Mixed Reality (MR) integrates virtual objects with the real world, offering potential but raising concerns about misuse through dark patterns. This study explored the effects of four dark patterns, adapted from prior research, and applied to MR across three targets: places, products, and people. In a two-factorial within-subject study with 74 participants, we analyzed 13 videos simulating MR experiences during a city walk. Results show that all dark patterns significantly reduced user comfort, increased reactance, and decreased the intention to use MR glasses, with the most disruptive effects linked to personal or monetary manipulation. Additionally, the dark patterns of Emotional and Sensory Manipulation and Hiding Information produced similar impacts on the user in MR, suggesting a re-evaluation of current classifications to go beyond deceptive design techniques. Our findings highlight the importance of developing ethical design guidelines and tools to detect and prevent dark patterns as immersive technologies continue to evolve.

As vehicle automation advances to SAE Levels 3 to 5, transitioning driving control from human to system, ensuring automated vehicles (AVs) align with user preferences becomes a challenge. Natural interaction emerges as a common goal, offering ways to convey user interests in a user-friendly manner. However, technical, legal, or design constraints may prevent fulfilling these preferences, leading to potential conflicts. Through an online survey (N=50), potential driver-passenger conflicts and their handling strategies were explored. Subsequently, in a Virtual Reality study (N=14), we applied identified strategies (ranging from distracting to motivating and adhering to social norms) to user-AV interactions using a state-of-the-art language model (GPT-4 Turbo) primed with the strategies to simulate realistic dialogues. Additionally, adaptive communication was compared to non-adaptive communication. Our findings reveal a preference for adaptive communication. Yet, despite using advanced modeling, accurately predicting user interactions remained challenging, with users often trying to outsmart the AI.

Automated vehicles are expected to improve traffic safety and efficiency. One approach to achieve this is via platooning, that is, (automated) vehicles can drive behind each other at very close proximity to reduce air resistance. However, this behavior could lead to difficulties in mixed traffic, for example, when manual drivers try to enter a highway. Therefore, we report the results of a within-subject Virtual Reality study (N=29) evaluating different platoon behaviors (single vs. multiple, i.e., four, gaps) and communication strategies (HUD, AR, attached displays). Results show that AR communication reduced mental workload, improved perceived safety, and a single big gap led to the safest merging behavior. Our work helps to incorporate novel behavior enabled by automation into general traffic better.

Gaze interaction is a promising interaction method to increase variety, challenge, and fun in games.We present “Shed Some Fear”, a 2D platform game including numerous eye-gaze-based interactions. “Shed Some Fear” includes control with eye-gaze and traditional keyboard input. The eye-gaze interactions are partially based on eye exercises reducing digital eye strain but also on employing peripheral vision. By employing eye-gaze as a necessary input mechanism, we explore the effects on and tradeoffs between user enjoyment and digital eye strain in a five-day longitudinal between-subject study (N=17) compared to interaction with a traditional mouse. We found that perceived competence was significantly higher with eye gaze interaction and significantly higher internal eye strain. With this work, we contribute to the not straightforward inclusion of eye tracking as a useful and fun input method for games.

Hybrid board games (HBGs) augment their analog origins digitally (e.g., through apps) and are an increasingly popular pastime activity. Continuous world and character development and customization, known to facilitate engagement in video games, remain rare in HBGs. If present, they happen digitally or imaginarily, often leaving physical aspects generic. We developed DungeonMaker, a fabrication-augmented HBG bridging physical and digital game elements: 1) the setup narrates a story and projects a digital game board onto a laser cutter; 2) DungeonMaker assesses player-crafted artifacts; 3) DungeonMaker’s modified laser head senses and moves player- and non-player figures, and 4) can physically damage figures. An evaluation (n = 4 × 3) indicated that DungeonMaker provides an engaging experience, may support players’ connection to their figures, and potentially spark novices’ interest in fabrication. DungeonMaker provides a rich constellation to play HBGs by blending aspects of craft and automation to couple the physical and digital elements of an HBG tightly.

Extended Reality (XR) allows in-situ previewing of designs to be manufactured through Personal Fabrication (PF). These in-situ interactions exhibit advantages for PF, like incorporating the environment into the design process. However, design-for-fabrication in XR often happens through either highly complex 3D-modeling or is reduced to rudimentary adaptations of crowd-sourced models. We present pARam, a tool combining parametric designs (PDs) and XR, enabling in-situ configuration of artifacts for PF. In contrast to modeling- or search-focused approaches, pARam supports customization through embodied and practical inputs (e.g., gestures, recommendations) and evaluation (e.g., lighting estimation) without demanding complex 3D-modeling skills. We implemented pARam for HoloLens 2 and evaluated it (n = 20), comparing XR and desktop conditions. Users succeeded in choosing context-related parameters and took their environment into account for their configuration using pARam. We reflect on the prospects and challenges of PDs in XR to streamline complex design methods for PF while retaining suitable expressivity.

As automated vehicles become more widespread but lack a driver to communicate in uncertain situations, external communication, for example, via LEDs or displays, is evaluated. However, the concepts are mostly evaluated in simple scenarios, such as one person trying to cross in front of one automated vehicle. The traditional empirical approach fails to study the large-scale effects of these in this not-yet-real scenario. Therefore, we built PedSUMO, an enhancement to SUMO for the simulacra of automated vehicles' effects on public traffic, specifically how pedestrian attributes affect their respect for automated vehicle priority at unprioritized crossings. We explain the algorithms used and the derived parameters relevant to the crossing. We open-source our code under https://github.com/M-Colley/pedsumo and demonstrate an initial data collection and analysis of Ingolstadt, Germany.

Figure 1: VRCreatIn is a virtual reality (VR) 3D content creation solution that supports the process of 3D scene creation (a) starting from ideation and going beyond to modeling (c), lighting, and texturing (a, d).It is based on a design space focusing on in-situ interaction for unconstrained 3D mid-air and constrained 2D surface-based interaction using pen and tablet (b).

Environments can affect mood, motivation, and productivity. Green spaces, for example, are known to have calming effects on people’s moods. In virtual reality (VR), we could take advantage of these effects, as we have full visual control over the environment. In this paper, we explore how such potential effects caused by the environment impact performance and user experience (UX) when playing an exergame. We created four environments differing in their level of detail and visual realism.: (1) a white room, (2) outer space, (3) an abstract space, and (4) a forest environment. In a user study (N=26) in which participants played an exergame in all four environments, we found evidence that VEs influence enjoyment and performance. The simulation of green spaces or abstract VEs with enjoyable background sounds has a particularly positive impact. We discuss how environmental features impact performance and UX and present promising avenues for future work investigating specific parts of environmental features.

Sedentary occupations and recreational activities carried out primarily while seated promote extended time periods spent in unhealthy sitting postures, contributing to physical and mental health issues. While apps and reminders can be effective, they often fail to sustain enjoyment and motivation or do not target stationary settings. In our work, we investigate whether sedentary waiting periods could be broken up through gentle full-body movements via full-body interactions on a large touchwall instead of remaining seated or standing. In a mixed-methods study (N=18), we compared a Match-3 game played (1) on a full-body touchwall, (2) on a smartphone standing, and (3) on a smartphone sitting, investigating user experience, performance, and acceptance. The touchwall game subtly motivated people to move, stretch and bend their bodies without performance loss while enjoying the game compared to the smartphone conditions. We suggest that full-body touchwall interaction has the potential to fill occasional waiting time while encouraging breaking up sedentary behavior.

Stories have long captivated the human imagination with narratives that enrich our lives. Traditional storytelling methods are often static and not designed to adapt to the listener’s environment, which is full of dynamic changes. For instance, people often listen to stories in the form of podcasts or audiobooks while traveling in a car. Yet, conventional in-car storytelling systems do not embrace the adaptive potential of this space. The advent of generative AI is the key to creating content that is not just personalized but also responsive to the changing parameters of the environment. We introduce a novel system for interactive, real-time story narration that leverages environment and user context in correspondence with estimated arrival times to adjust the generated story continuously. Through two comprehensive real-world studies with a total of 30 participants in a vehicle, we assess the user experience, level of immersion, and perception of the environment provided by the prototype. Participants’ feedback shows a significant improvement over traditional storytelling and highlights the importance of context information for generative storytelling systems.

Minor violations of traffic regulations are common today and partially socially accepted. Automated vehicles (AVs), however, will be obliged to keep to the letter of the law. This can lead to situations where user requests cause the AV to reach its legal boundaries, creating novel user-vehicle conflicts. To investigate whether traffic-violating driver interests are transferred to the automated context, we conducted an online survey with three conflict-prone scenarios (N=49). The results indicate that legally compliant AV behavior is desired but that users would intervene in the vehicle’s behavior to enforce interests. In a subsequent Virtual Reality study (N=30), we evaluated the effect of legal boundary-handling strategies (Responsibility and Control Shift, Responsibility Shift, No Shift) and other traffic participants’ violating traffic regulations on behavior, conflict, and trust in a legally conflict-prone parking scenario. Results show that conflict is perceived significantly higher in all strategies compared to the manual baseline, while situational trust in the vehicle is higher in the automated conditions but independent of the handling strategy.

Object manipulation in 3D space, meaning translating, rotating, and scaling, is ubiquitous in virtual reality (VR), and several interaction techniques have been developed in the past to optimize the task performance and usability. However, preliminary research indicates that individual spatial abilities also have an impact. Yet, it was never investigated if users’ spatial abilities influence VR object manipulation. We assessed this in a user study (N=66) using 21 manipulation tasks defined in a Fitts’ law-related approach. As interaction techniques, we chose gizmos for simultaneously manipulating 1 and 3 degrees of freedom (DOF) and a handle bar metaphor for 7 DOF. Higher spatial abilities resulted in significantly shorter task completion time and more targeted manipulations, while task accuracy was unaffected. However, an optimized interaction technique could compensate individual disadvantages. We propose seven guidelines on spatial abilities in interaction technique design and research to personalize and improve VR applications.

Automotive user interface (AUI) evaluation becomes increasingly complex due to novel interaction modalities, driving automation, heterogeneous data, and dynamic environmental contexts. Immersive analytics may enable efficient explorations of the resulting multilayered interplay between humans, vehicles, and the environment. However, no such tool exists for the automotive domain. With AutoVis, we address this gap by combining a non-immersive desktop with a virtual reality view enabling mixed-immersive analysis of AUIs. We identify design requirements based on an analysis of AUI research and domain expert interviews (N=5). AutoVis supports analyzing passenger behavior, physiology, spatial interaction, and events in a replicated study environment using avatars, trajectories, and heatmaps. We apply context portals and driving-path events as automotive-specific visualizations. To validate AutoVis against real-world analysis tasks, we implemented a prototype, conducted heuristic walkthroughs using authentic data from a case study and public datasets, and leveraged a real vehicle in the analysis process.

Novel display technologies, such as lightfield displays, have become increasingly available. In the automotive domain, these are already in use or are to be used in the future. However, their effect on the user is yet to be explored. Therefore, we conducted a within-subject study (N=15) comparing a baseline visualization of information about automated vehicle functionality with it being visualized either via a LumePad or a LookingGlass display. Interestingly, we found almost no significant differences, thus, indicating that the display technology is less relevant for conveying automated functionality.

Augmented Reality (AR) can alter environments and steer attention. While prior work dominantly focuses on exploring performances of augmentations, this work aims to understand the societal impact of AR in complex social situations. Focusing on prosocial helping behaviour, we created two scenarios and designed five augmentations aiming to motivate a user to help. We wanted to understand (1) the impact on situation perception and (2) the impact on the social structure. In an online video experiment (n=294), we found that augmenting can impact anxiety about the situation and significantly increase the perceived reason to help being directed towards the augmentation. Similarly, we found that the helped rated the "reason" and "thankfulness" significantly higher towards AR than the helper, creating a disagreement around agency and responsibility. We discuss the implications of AR in complex social structures and how responsibility and agency will become important when embedding AR in our social lives.

In highly automated vehicles, passengers can engage in non-driving-related activities. Additionally, the technical advancement allows for novel interaction possibilities such as voice, gesture, gaze, touch, or multimodal interaction, both to refer to in-vehicle and outside objects (e.g., thermostat or restaurant). This interaction can be characterized by levels of urgency (e.g., based on late detection of objects) and cognitive load (e.g., because of watching a movie or working). Therefore, we implemented a Virtual Reality simulation and conducted a within-subjects study with N=11 participants evaluating the effects of urgency and cognitive load on modality usage in automated vehicles. We found that while all modalities were possible to use, participants relied on touch the most. This was followed by gaze, especially for external referencing. This work helps to further understand multimodal interaction and the requirements this poses on natural interaction in (automated) vehicles.

Tile-based locomotion (TBL) is a popular locomotion technique for computer, console, and board games. However, despite its simplicity and unconventional movement, the transfer of TBL to virtual reality (VR) as a game platform remains unexplored. To fill this gap, we introduce TBL for VR on the example of two techniques: a controller and a feet-based one. In a first user study, we evaluated the usability and acceptance of the techniques compared to teleportation and touchpad locomotion. In a second exploratory user study, we evaluated the user experience of both TBL techniques in a maze and a museum scenario. The findings show that both techniques provide enjoyment and acceptable usability by creating either a relaxing (controller-based) or a physically active (feet-based) solution. Finally, our results highlight that TBL techniques work particularly well for small, constrained spaces that allow users to focus on exploring details in the nearby environment (important for games) in contrast to large open spaces that require faster locomotion, like teleportation.

Today's social media (SM) platforms are toolkits consisting of features with different use cases, some strongly related to habitual and regretful use. Especially Infinite Scrolling (IS) has been reported to make users feel like they are being caught in a loop, regretfully elongating SM sessions. We investigated and defined this loop while unveiling the processes that make users break it. Based on a one-week-long field study (N=46), we unfolded and categorized general reasons for leaving social media and related those to IS. In light of our findings, we argue that SM interventions should not only focus on the app but incorporate the user's context, as most reasons to break SM sessions were not related to the app but the user's general context. Our findings and prior work also indicate the coexistence of multiple loops, which we define as inner (intra-session) loops surrounded by an outer (habitual) loop.

3D-printers enable end-users to design and fabricate unique physical artifacts but maintain an increased entry barrier and friction. End users must design tangible artifacts through intangible media away from the main problem space (ex-situ) and transfer spatial requirements to an abstract software environment. To allow users to evaluate dimensions, balance, or fit early and in-situ, we developed BrickStARt, a design tool using tangible construction blocks paired with a mixed-reality headset. Users assemble a physical block model at the envisioned location of the fabricated artifact. Designs can be tested tangibly, refined, and digitally post-processed, remaining continuously in-situ. We implemented BrickStARt using a Magic Leap headset and present walkthroughs, highlighting novel interactions for 3D-design. In a user study (n=16), first-time 3D-modelers succeeded more often using BrickStARt than Tinkercad. Our results suggest that BrickStARt provides an accessible and explorative process while facilitating quick, tangible design iterations that allow users to detect physics-related issues (e.g., clearance) early on.

Automated vehicles are expected to substitute driver-pedestrian communication via LED strips or displays. This communication is expected to improve trust and the crossing process in general. However, numerous factors such as other pedestrians’ behavior, perceived time pressure, or previous experience influence crossing decisions. Therefore, we report the results of a triply subdivided Virtual Reality study (N=18) evaluating these. Results show that external communication was perceived as hedonically pleasing, increased perceived safety and trust, and also that pedestrians’ behavior affected participants’ behavior. A timer did not alter crossing behavior, however, repeated exposure increased trust and reduced crossing times, showing a habituation effect. Our work helps better to integrate research on external communication in ecologically valid settings.

Pair-learning is beneficial for learning outcome, motivation, and social presence, and so is virtual reality (VR) by increasing immersion, engagement, motivation, and interest of students. Nevertheless, there is a research gap if the benefits of pair-learning and VR can be combined. Furthermore, it is not clear which influence it has if only one or both peers use VR. To investigate these aspects, we implemented two types of VR pair-learning systems, a symmetric system with both peers using VR and an asymmetric system with one using a tablet. In a user study (N=46), the symmetric system statistically significantly provided higher presence, immersion, player experience, and lower intrinsic cognitive load, which are all important for learning. Symmetric and asymmetric systems performed equally well regarding learning outcome, highlighting that both are valuable learning systems. We used these findings to define guidelines on how to design co-located VR pair-learning applications, including characteristics for symmetric and asymmetric systems.

Voice assistants (VAs) are present in homes, smartphones, and cars. They allow users to perform tasks without graphical or tactile user interfaces, as they are designed for natural language interaction. However, we found that currently, VAs are emulating human behavior by responding in complete sentences, limiting the design options, and preventing VAs from meeting their full potential as a utilitarian tool. We implemented a VA that handles requests in three response styles: two differing short keyword-based response styles and a full-sentence baseline. In a user study, 72 participants interacted with our VA by issuing eight requests. Results show that the short responses were perceived similarly useful and likable while being perceived as more efficient, especially for commands, and sometimes better to comprehend than the baseline. To achieve widespread adoption, we argue that VAs should be customizable and adapt to users instead of always responding in full sentences.

Social Media (SM) has shown that we adapt our communication and disclosure behaviors to available technological opportunities. Head-mounted Augmented Reality (AR) will soon allow to effortlessly display the information we disclosed not isolated from our physical presence (e.g., on a smartphone) but visually attached to the human body. In this work, we explore how the medium (AR vs. Smartphone), our role (being augmented vs. augmenting), and characteristics of information types (e.g., level of intimacy, self-disclosed vs. non-self-disclosed) impact the users’ comfort when displaying personal information. Conducting an online survey (N=148), we found that AR technology and being augmented negatively impacted this comfort. Additionally, we report that AR mitigated the effects of information characteristics compared to those they had on smartphones. In light of our results, we discuss that information augmentation should be built on consent and openness, focusing more on the comfort of the augmented rather than the technological possibilities.

Personal fabrication is made more accessible through repositories like Thingiverse, as they replace modeling with retrieval. However, they require users to translate spatial requirements to keywords, which paints an incomplete picture of physical artifacts: proportions or morphology are non-trivially encoded through text only. We explore a vision of in-situ spatial search for (future) physical artifacts, and present ShapeFindAR, a mixed-reality tool to search for 3D models using in-situ sketches blended with textual queries. With ShapeFindAR, users search for geometry, and not necessarily precise labels, while coupling the search process to the physical environment (e.g., by sketching in-situ, extracting search terms from objects present, or tracing them). We developed ShapeFindAR for HoloLens 2, connected to a database of 3D-printable artifacts. We specify in-situ spatial search, describe its advantages, and present walkthroughs using ShapeFindAR, which highlight novel ways for users to articulate their wishes, without requiring complex modeling tools or profound domain knowledge.

Interacting with a group of people requires to direct the attention of the whole group, thus requires feedback about the crowd’s attention. In face-to-face interactions, head and eye movements serve as indicator for crowd attention. However, when interacting online, such indicators are not available. To substitute this information, gaze visualizations were adapted for a crowd scenario. We developed, implemented, and evaluated four types of visualizations of crowd attention in an online study with 72 participants using lecture videos enriched with audience’s gazes. All participants reported increased connectedness to the audience, especially for visualizations depicting the whole distribution of gaze including spatial information. Visualizations avoiding spatial overlay by depicting only the variability were regarded as less helpful, for real-time as well as for retrospective analyses of lectures. Improving our visualizations of crowd attention has the potential for a broad variety of applications, in all kinds of social interaction and communication in groups.

Webcam-based eye-tracking promises easy and quick data collection without the need for specific or additional eye-tracking hardware. This makes it especially attractive for educational research, in particular for modern formats, such as MOOCs. However, in order to fulfill its promises, webcam-based eye tracking has to overcome several challenges, most importantly, varying spatial and temporal resolutions. Another challenge that the educational domain faces especially, is that typically individual students are of interest in contrast to average values. In this paper, we explore whether an attention measure that is based on eye movement synchronicity of a group of students can be applied with unreliably-sampled data. Doing so we aim to reproduce earlier work that showed that, on average, eye movement synchronicity can predict performance in a comprehension quiz. We were not able to reproduce the findings with unreliably-sampled data, which highlights the challenges that lie ahead of webcam-based eye tracking in practice.

Personal fabrication empowers users to create objects increasingly easier and faster. This continuous decrease in effort evokes a speculative scenario of Ephemeral Fabrication (EF), enabled and amplified by emerging paradigms of mobile, wearable, or even body-integrated fabrication. EF yields fast, temporary, in-situ solutions for everyday problems (e.g., creating a protective skin, affixing a phone). Users solely create those, since the required effort is negligible. We present and critically reflect on the EF scenario, by exploring current trends in research and building a body-worn fabrication device. EF is a plausible extrapolation of current developments, entailing both positive (e.g., accessibility) and negative implications (e.g., unsustainability). Using speculative design methodology to question the trajectory of personal fabrication, we argue that to avert the aftermath of such futures, topics like sustainability can not remain an afterthought, but rather be situated in interactions themselves: through embedded constraints, conscious material choice, and constructive embedding of ephemerality.

Automated vehicles are expected to substitute or even improve driver-driver communication, for example, via LED strips or displays. Numerous situations exist where ambiguities have to be resolved via gestures or implicit communication (i.e., movement). An already demanding situation is the unsignalized four-way intersection. Additionally, Vehicle-To-Everything technology enables automated vehicles to perform maneuvers impossible before such as blocking an intersection to safely let an emergency vehicle pass. Therefore, we report the results of a within-subject Virtual Reality study (N=17) evaluating these two scenarios. Results show that external communication increased perceived safety and reduced mental workload, and also that the novel behavior confused participants. Our work helps better to integrate external communication in settings with manual drivers.

Automated vehicles will change the interaction with the user drastically. While freeing the user of the driving task for most of the journey, the "final 100 meters problem'', directing the vehicle to the final parking spot, could require human intervention. Therefore, we present a classification of interaction concepts for automated vehicles based on modality and interaction mode. In a subsequent Virtual Reality study (N=16), we evaluated sixteen interaction concepts. We found that the medially abstracted interaction mode was consistently rated most usable over all modalities (joystick, speech, gaze, gesture, and tablet). While the steering wheel was still preferred, our findings indicate that other interaction concepts are usable if the steering wheel were unavailable.

Infrastructure-mounted sensors that monitor roads can provide essential information for manual drivers and automated vehicles, e.g., positions of other vehicles occluded by buildings. However, human drivers and passengers have to trust and accept their use. This raises the question of how trust can be increased in such a scenario. One important factor for this is understanding the available information, including its quality and, for passengers of automated vehicles, the actions planned based on it. For this, augmented reality is a promising visualization technology because it can present the relevant information integrated into the physical world. Thus, this work develops a taxonomy of augmented reality visualizations for connected automated and manual driving. It is intended to classify and compare existing visualizations, identify novel visualizations, and provide a common language for discussions. The use case infrastructure-supported automated driving is explored by suggesting augmented reality visualizations to inform passengers of automated vehicles and are intended to increase trust. They present information available from infrastructure and onboard sensors as well as the driving decisions based on it. Finally, we evaluated the visualizations' influence on trust in an automated vehicle by conducting a driving simulator study (N=18). Results indicate a high dependency of trust on presenting driving decisions and information on road users but less on location-specific information.

The inclusion of in-vehicle sensors and increased intention and state recognition capabilities enable implicit in-vehicle interaction. Starting from a systematic literature review (SLR) on implicit in-vehicle interaction, which resulted in 82 publications, we investigated state and intention recognition methods based on (1) their used modalities, (2) their underlying level of automation, and (3) their considered interaction focus. Our SLR revealed a research gap addressing implicit interaction in highly automated vehicles (HAVs). Therefore, we discussed how the requirements for implicit state and intention recognition methods and interaction based on them are changing in HAVs. With this, open questions and opportunities for further research in this area were identified.

Abstract Large high-resolution displays (LHRDs) provide an enabling technology to achieve immersive, isometrically registered, virtual environments. It has been shown that LHRDs allow better size judgments, higher collaboration performance, and shorter task completion times. This paper presents novel insights into human size perception using large-scale floor displays, in particular in-depth evaluations of size judgment accuracy, precision, and task completion time. These investigations have been performed in the context of six, novel applications in the domain of automotive production planning. In our studies, we used a 54-sqm sized LED floor and a standard tablet visualizing relatively scaled and true to scale 2D content, which users had to estimate using different aids. The study involved 22 participants and three different conditions. Results indicate that true to scale floor visualizations reduce the mean absolute percentage error of spatial estimations. In all three conditions, we did not find the typical overestimation or underestimation of size judgments.

Digital eye strain (DES), caused by prolonged exposure to digital screens, stresses the visual system and negatively affects users’ well-being and productivity. While DES is well-studied in computer displays, its impact on users of virtual reality (VR) head-mounted displays (HMDs) is largely unexplored—despite that some of their key properties (e.g., the vergence-accommodation conflict) make VR-HMDs particularly prone. This work provides the first comprehensive investigation into DES in VR HMDs. We present results from a survey with 68 experienced users to understand DES symptoms in VR-HMDs. To help address DES, we investigate eye exercises resulting from survey answers and blue light filtering in three user studies (N = 71). Results demonstrate that eye exercises, but not blue light filtering, can effectively reduce DES. We conclude with an extensive analysis of the user studies and condense our findings in 10 key challenges that guide future work in this emerging research area.

Automated vehicles are about to enter the mass market. However, such systems regularly meet limitations of varying criticality. Even basic tasks such as Object Identification can be challenging, for example, under bad weather or lighting conditions or for (partially) occluded objects. One common approach is to shift control to manual driving in such circumstances, however, post-automation effects can occur in these control transitions. Therefore, we present ORIAS, a system capable of asking the driver to (1) identify/label unrecognized objects or to (2) select an appropriate action to be automatically executed. ORIAS extends the automation capabilities, prevents unnecessary takeovers, and thus reduces post-automation effects. This work defines the capabilities and limitations of ORIAS and presents the results of a study in a driving simulator (N=20). Results indicate high usability and input correctness.

Autonomous vehicles (AVs) are expected to communicate to vulnerable road users as a substitution of, for example, driver-pedestrian communication, leading to increased safety and acceptance. This communication is currently one-directional, i.e., from the AV to the pedestrian. However, today’s communication between drivers and pedestrians in crossing scenarios is bidirectional. Pedestrians gesture “thank you’’ or wave drivers through in case they do not want to cross. Human drivers often acknowledge this, for example, with a nod. We present an experiment in Virtual Reality (N=20), in which the effect of such acknowledgment of the AVs via its external communication is investigated for the two described scenarios and concerning pedestrian presence. Results show that such feedback is perceived as highly necessary, depends on the scenario, and improves the perceived intelligence of the AV, confirming a Halo-Effect.

Traffic behavior and its perception is shaped by various factors such as vehicle color or size. Decals are used to express information about the owner’s beliefs or are intended to be funny. In the future, with external displays on (automated) vehicles, individualized customization could be even more pronounced. While some research looked at the messages these decals convey, it is unclear how these decals influence the perception of surrounding drivers on the operator of the vehicle. We gathered data on decals in 29 cities in 8 countries. A thematic analysis unveiled 17 dominant themes among decals. Subsequently, we investigated effects of decals of 9 supra-regional common themes in an online study (N=64) finding that participants attributed different characteristics to the driver of a vehicle with a decal based on the type of decal and the participants’ country of origin. Additionally, a Virtual Reality study (N=16) revealed diverse opinions on future usage of such personalization options.

Autonomous vehicles could improve mobility, safety, and inclusion in traffic. While this technology seems within reach, its successful introduction depends on the intended user’s acceptance. A substantial factor for this acceptance is trust in the autonomous vehicle’s capabilities. Visualizing internal information processed by an autonomous vehicle could calibrate this trust by enabling the perception of the vehicle’s detection capabilities (and its failures) while only inducing a low cognitive load. Additionally, the simultaneously raised situation awareness could benefit potential take-overs. We report the results of two comparative online studies on visualizing semantic segmentation information for the human user of autonomous vehicles. Effects on trust, cognitive load, and situation awareness were measured using a simulation (N=32) and state-of-the-art panoptic segmentation on a pre-recorded real-world video (N=41). Results show that the visualization using Augmented Reality increases situation awareness while remaining low cognitive load.

Based on a systematic literature review of more than 300 papers published over the last 10 years, we provide indicators that the simulator sickness questionnaire (SSQ) is extensively used and widely accepted as a general discomfort measure in virtual reality (VR) research – although it actually only accounts for one category of symptoms. This results in important other categories (digital eye strain (DES) and ergonomics) being largely neglected. To contribute to a more comprehensive picture of discomfort in VR head-mounted displays, we further conducted an online study (N=352) on the severity and relevance of all three symptom categories. Most importantly, our results reveal that symptoms of simulator sickness are significantly less severe and of lower prevalence than those of DES and ergonomics. In light of these findings, we critically discuss the current use of SSQ as the only discomfort measure and propose a more comprehensive factor model that also includes DES and ergonomics.

Recent automotive research often focuses on automated driving, including the interaction between automated vehicles (AVs) and so-called “vulnerable road users” (VRUs). While road safety statistics and traffic psychology at least define VRUs as pedestrians, cyclists, and motorcyclists, many publications on human-vehicle interaction use the term without even defining it. The actual target group remains unclear. Since each group already poses a broad spectrum of research challenges, a one-fits-all solution seems unrealistic and inappropriate, and a much clearer differentiation is required. To foster clarity and comprehensibility, we propose a literature-based taxonomy providing a structured separation of (vulnerable) road users, designed to particularly (but not exclusively) support research on the communication between VRUs and AVs. It consists of two conceptual hierarchies and will help practitioners and researchers by providing a uniform and comparable set of terms needed for the design, implementation, and description of HCI applications.

Virtual reality (VR) multiplayer games increasingly use asymmetry (e.g., differences in a person’s capability or the user interface) and resulting interdependence between players to create engagement even when one player has no access to a head-mounted display (HMD). Previous work shows this enhances player experience (PX). Until now, it remains unclear whether and how an asymmetric game design with interdependences creates comparably enjoyable PX for both an HMD and a non-HMD player. In this work, we designed and implemented an asymmetric VR game (different in its user interface) with two types of interdependence: strategic (difference in game information/player capability) and biometric (difference in player’s biometric influence). Our mixed-methods user study (N=30) shows that asymmetries positively impact PX for both player roles, that interdependence strongly affects players’ perception of agency, and that biometric feedback—while subjective—is a valuable game mechanic.

Augmented Reality (AR) glasses equip users with the tools to modify the visual appearance of their surrounding environment. This might severely impact interpersonal communication, as the conversational partners will no longer share the same visual perception of reality. Grounded in color-in-context theory, we present a potential AR application scenario in which users can modify the color of the environment to achieve subconscious benefits. In a consecutive online survey (N=64), we measured the user’s comfort, acceptance of altering and being altered, and how it is impacted by being able to perceive or not perceive the alteration. We identified significant differences depending on (1) who or what is the target of the alteration, (2) which body part is altered, and (3) which relationship the conversational partners share. In light of our quantitative and qualitative findings, we discuss ethical and practical implications for future devices and applications that employ visual alterations.

In (partially) automated vehicles, users will sometimes have to take over control due to system failure or reach of an operational driving domain end. In such scenarios, the user becomes the driver and has to quickly gain situational awareness. With advanced sensory, an automated vehicle could aid the user in building situational awareness by providing information. A literature analysis found no commonly conveyed information. Therefore, to evaluate the effects of four different abstraction levels (high, medium-high, medium, and low) and used modalities (visual vs. visual+auditory) on situational awareness and accompanying usability scores, we evaluated eight systems and a baseline without information display. In the between-subjects online monitor-based study (N=225), we found that while subjective measures are higher and a warning is required, providing abstract information does not improve objective situational awareness, and only being provided with visual information was preferred.

Pedestrians are very vulnerable road users. Autonomous vehicles driving on the street are expected to reduce the risks for pedestrians. However, traffic will include manually driven vehicles for the next decades; therefore, the risk remains and can even increase due to pedestrians’ overreliance on technology. Thus, we propose to use autonomous vehicles parked at the side of the road to continually survey its surroundings and to issue warnings to pedestrians in potentially risky situations. Findings of a Virtual Reality (N=20) study show that participants preferred the communication on the vehicle compared to a communication on the sidewalk, that the visualization grabbed participants’ attention, and that overall crossing time was not significantly affected. This concept highlights the potential autonomous vehicles could have in making traffic safer, even while being parked.

Players can get stuck in video games, which impedes their process to their goal and results in unfavorable outcomes like negative emotions, impediments of flow, and obstacles for learning. Currently, it is not easily possible to assess if a player is stuck, as no widely accepted definition of "being stuck" in games exists. We conducted 13 expert interviews and a systematic literature review with 104 relevant papers selected from 4022 candidates. We present a definition of "being stuck" that conceptualizes the state as a continuum and contextualize it within related concepts. Our stuck continuum can be applied to regulate the player's stuck level. We propose a taxonomy of measures that are useful for the detection of the level of stuckness and discuss the effectiveness of countermeasures. Our stuck concept is crucial for game developers creating an optimal player experience in games.

Many people utilize audio equipment to escape from noises around them, leading to the desired isolation but also dangerously reduced awareness. Mediation of sounds through smarter headphones (e.g., hearables) could address this by providing non-uniform interaction with sounds while retaining a comfortable, yet informative soundscape. In a week-long event sampling study (n = 12), we found that users mostly desire muting or a distinct "quiet-but-audible" volume for sound sources. A follow-up study (n = 12) compared a reduced interaction granularity with a continuous one in VR. Usability and workload did not differ significantly for the two granularities but a set of four states can be considered sufficient for most scenarios, namely: ''muted'', ''quieter'', ''louder'' and ''unchanged'', allowing for smoother interaction flows. We provide implications for the design of interactive auditory mediated reality systems enabling users to be safe, comfortable and less isolated from their surroundings, while re-gaining agency over their sense of hearing.

Autonomous vehicles carry the potential to greatly improve mobility and safety in traffic. However, this technology has to be accepted and of value for the intended users. One challenge on this way is the detection and recognition of pedestrians and their intentions. While there are technological solutions to this problem, there seems to be no research on how to make this information transparent to the user in order to calibrate the user’s trust. Our work presents a comparative study of 5 visualization techniques with Augmented Reality or tablet-based visualization technology and two or three information clarity states of pedestrian intention in the context of highly automated driving. We investigated these in a user study in Virtual Reality (N=15). We found that such a visualization was rated reasonable, necessary, and that especially the Augmented Reality-based version with three clarity states was preferred.

Automated vehicles will change the trucking industry as human drivers become more absent. In crossing scenarios, external communication concepts are already evaluated to resolve potential issues. However, automated delivery poses unique communication problems. One specific situation is the delivery to the curb with the truck remaining partially on the street, blocking sidewalks. Here, pedestrians have to walk past the vehicle with reduced sight, resulting in safety issues. To address this, we conducted a literature survey revealing the lack of addressing external communication of automated vehicles in situations other than crossings. Afterwards, a study in Virtual Reality (N=20) revealed the potential of such communication. While the visualization (e.g., arrows or text) of whether it is safe to walk past the truck only played a minor part, the information of being able to safely walk past was highly appreciated. This shows that external communication concepts carry great potential besides simple crossing scenarios.

Autonomous vehicles are on the verge of entering the mass market. Communication between these vehicles with vulnerable road users could increase safety and ease their introduction by helping to understand the vehicle’s intention. Numerous communication modalities and messages were proposed and evaluated. However, these explorations do not account for the factors described in communication theory. Therefore, we propose a two-part design space consisting of a concept part with 3 dimensions and a situation part with 6 dimensions based on a literature review on communication theory and a focus group with experts (N=4) on communication. We found that most work until now does not address situation-specific aspects of such communication.

Virtual and augmented reality head-mounted displays (HMDs) are currently heavily relying on spatially tracked input devices (STID) for interaction. These STIDs are all prone to the phenomenon that a discrete input (e.g. button press) will disturb the position of the tracker, resulting in a different selection point during ray-cast interaction (Heisenberg Effect of Spatial Interaction). Besides the knowledge of its existence, there is currently a lack of a deeper understanding of its severity, structure and impact on throughput and angular error during a selection task. In this work, we present a formal evaluation of the Heisenberg effect and the impact of body posture, arm position and STID degrees of freedom on its severity. In a Fitt's Law inspired user study (N=16), we found that the Heisenberg effect is responsible for 30.45% of the overall errors occurring during a pointing task, but can be reduced by 25.4% using a correction function.

One of the great benefits of virtual reality (VR) is the implementation of features that go beyond realism. Common "unrealistic" locomotion techniques (like teleportation) can avoid spatial limitation of tracking, but minimize potential benefits of more realistic techniques (e.g. walking). As an alternative that combines realistic physical movement with hyper-realistic virtual outcome, we present JumpVR, a jump-based locomotion augmentation technique that virtually scales users' physical jumps. In a user study (N=28), we show that jumping in VR (regardless of scaling) can significantly increase presence, motivation and immersion compared to teleportation, while largely not increasing simulator sickness. Further, participants reported higher immersion and motivation for most scaled jumping variants than forward-jumping. Our work shows the feasibility and benefits of jumping in VR and explores suitable parameters for its hyper-realistic scaling. We discuss design implications for VR experiences and research.

People with vision impairments (VIP) are among the most vulnerable road users in traffic. Autonomous vehicles are believed to reduce accidents but still demand some form of external communication signaling relevant information to pedestrians. Recent research on the design of vehicle-pedestrian communication (VPC) focuses strongly on concepts for a non-disabled population. Our work presents an inclusive user-centered design for VPC, beneficial for both vision impaired and seeing pedestrians. We conducted a workshop with VIP (N=6), discussing current issues in road traffic and comparing communication concepts proposed by literature. A thematic analysis unveiled two important themes: number of communicating vehicles and content (affecting duration). Subsequently, we investigated these in a second user study in virtual reality (N=33, 8 VIP) comparing the VPC between groups of abilities. We found that trust and understanding is enhanced and cognitive load reduced when all relevant vehicles communicate; high content messages also reduce cognitive load.

Sketching in virtual reality (VR) enhances perception and understanding of 3D volumes, but is currently a challenging task, as spatial input devices (e.g., tracked controllers) do not provide any scaffolding or constraints for mid-air interaction. We present VRSketchIn, a VR sketching application using a 6DoF-tracked pen and a 6DoF-tracked tablet as input devices, combining unconstrained 3D mid-air with constrained 2D surface-based sketching. To explore what possibilities arise from this combination of 2D (pen on tablet) and 3D input (6DoF pen), we present a set of design dimensions and define the design space for 2D and 3D sketching interaction metaphors in VR. We categorize prior art inside our design space and implemented a subset of metaphors for pen and tablet sketching in our prototype. To gain a deeper understanding which specific sketching operations users perform with 2D and which with 3D metaphors, we present findings of usability walkthroughs with six participants.

Natural navigation in VR is challenging due to spatial limitations. While Teleportation enables navigation within very small physical spaces and without causing motion sickness symptoms, it may reduce the feeling of presence and spacial awareness. Redirected walking (RDW), in contrast, allows users to naturally walk while staying inside a finite, but still very large, physical space. We present Telewalk, a novel locomotion approach that combines curvature and translation gains known from RDW research in a perceivable way. This combination enables Telewalk to be applied even within a physical space of 3m x 3m. Utilizing the head rotation as input device enables directional changes without any physical turns to keep the user always on an optimal circular path inside the real world while freely walking inside the virtual one. In a user study we found that even though motion sickness susceptible participants reported respective symptoms, Telewalk did result in stronger feelings of presence and immersion and was seen as more natural then Teleportation.

The accessibility of tools to model artifacts is one of the core driving factors for the adoption of Personal Fabrication. Subsequently, model repositories like Thingiverse became important tools in (novice) makers' processes. They allow them to shorten or even omit the design process, offloading a majority of the effort to other parties. However, steps like measurement of surrounding constraints (e.g., clearance) which exist only inside the users' environment, can not be similarly outsourced. We propose Mix&Match a mixed-reality-based system which allows users to browse model repositories, preview the models in-situ, and adapt them to their environment in a simple and immediate fashion. Mix&Match aims to provide users with CSG operations which can be based on both virtual and real geometry. We present interaction patterns and scenarios for Mix&Match, arguing for the combination of mixed reality and model repositories. This enables almost modelling-free personal fabrication for both novices and expert makers.

Display-based interfaces pose high demands on users’ eyes that can cause severe vision and eye problems, also known as digital eye strain (DES). Although these problems can become even more severe if the eyes are actively used for interaction, prior work on gaze-based interfaces has largely neglected these risks. We offer the first comprehensive account of DES in gaze-based interactive systems that is specifically geared to gaze interaction designers. Through an extensive survey of more than 400 papers published over the last 46 years, we first discuss the current role of DES in interactive systems. One key finding is that DES is only rarely considered when evaluating novel gaze interfaces and neglected in discussions of usability. We identify the main causes and solutions to DES and derive recommendations for interaction designers on how to guide future research on evaluating and alleviating DES.

Media (e.g. videos, images, and text) shared on social platforms such as Facebook and WeChat are often visually enriched through digital content (e.g. emojis, stickers, animal faces) increasing joy, personalization, and expressiveness. While voice messages (VMs) are experiencing a high frequent usage, they currently lack any form of digital augmentation. This work is the first to present and explore the concept of augmented VMs. Inspired by visual augmentations we designed and implemented an editor, allowing users to enhance VMs with background sounds, voice changers, and sound stickers. In a first evaluation (N = 15) we found that participants used augmentations frequently (2.73 per message on average) and rated augmented VMs to be expressive, personal and more fun than ordinary VMs. In a consecutive step, we analyzed the 45 augmented VMs recorded during the study and identified three distinct message types (decoration, composition and integrated) that inform about potential usage.

Adoption and use of smartphone-based asynchronous voice messaging has increased substantially in recent years. However, this communication channel has a strong tendency to polarize. To provide an understanding of this modality, we started by conducting an online survey (n=1,003) exploring who is using voice messages, their motives, and utilization. In a consecutive field study (n=6), we analyzed voice messaging behavior of six avid voice message users in a two-week field study, followed by semi-structured interviews further exploring themes uncovered in our survey. Conducting a thematic analysis, we identified four themes driving voice messaging usage: convenience, para-linguistic features, situational constraints and the receiver. Voice messaging helps to overcome issues of mobile communication, through ease of use, asynchronous implementation, and voices’ rich emotional context. It also was perceived as enabling more efficient communication, helps to handle secondary occupations, and better facilitates maintenance of close relationships. Despite the increased effort required to listen to a voice message, they complement communication with people we care about.

The design of graphical augmented reality (AR) markers requires compromise between the aesthetic appearance and tracking reliability. To investigate the topic, we created a virtual reality (VR) pipeline to evaluate marker performance, and validated it against real-world performance for a set of graphical AR markers. We report that, with the well known Vuforia framework and typical smartphone hardware, well designed 20 × 20 cm markers can be tracked at distances of up to 68 cm. We note that the number of feature points is particularly important to a marker’s angular performance.

Head-Mounted Displays (HMDs) are the dominant form of enabling Virtual Reality (VR) and Augmented Reality (AR) for personal use. One of the biggest challenges of HMDs is the exclusion of people in the vicinity, such as friends or family. While recent research on asymmetric interaction for VR HMDs has contributed to solving this problem in the VR domain, AR HMDs come with similar but also different problems, such as conflicting information in visualization through the HMD and projection. In this work, we propose ShARe, a modified AR HMD combined with a projector that can display augmented content onto planar surfaces to include the outside users (non-HMD users). To combat the challenge of conflicting visualization between augmented and projected content, ShARe visually aligns the content presented through the AR HMD with the projected content using an internal calibration procedure and a servo motor. Using marker tracking, non-HMD users are able to interact with the projected content using touch and gestures. To further explore the arising design space, we implemented three types of applications (collaborative game, competitive game, and external visualization). ShARe is a proof-of-concept system that showcases how AR HMDs can facilitate interaction with outside users to combat exclusion and instead foster rich, enjoyable social interactions.

Automated driving has the potential to reduce road fatalities. However, the public opinion to use automated driving can be described as skeptical. To increase the use of automated driving features, we investigate the persuasion principle of opt-out permission policies for enabling the automation, meaning automatically enabling the automation if users do not veto. In a driving simulator study (n = 19), participants drove on three different tracks (city, highway, rural). Three different interface concepts (opt-out, opt-in, control) were examined regarding their effects on automation use, trust, and acceptance. We found that an opt-out activation policy may increase automation usage for some participants. However, opt-out was perceived as more persuasive and more patronizing than the other conditions. Most importantly, opt-out can lead to mode confusion and therefore to dangerous situations. When such an opt-out policy is used in an automated vehicle, mode confusion must be addressed.

Augmented and virtual reality (AR/VR) has entered the mass market and, with it, will soon eye tracking as a core technology for next generation head-mounted displays (HMDs). In contrast to existing gaze interfaces, the 3D nature of AR and VR requires estimating a user's gaze in 3D. While first applications, such as foveated rendering, hint at the compelling potential of combining HMDs and gaze, a systematic analysis is missing. To fill this gap, we present the first design space for gaze interaction on HMDs. Our design space covers human depth perception and technical requirements in two dimensions aiming to identify challenges and opportunities for interaction design. As such, our design space provides a comprehensive overview and serves as an important guideline for researchers and practitioners working on gaze interaction on HMDs. We further demonstrate how our design space is used in practice by presenting two interactive applications: EyeHealth and XRay-Vision.

Motion synthesis approaches are widely used throughout different domains such as gaming, virtual crowds or simulation within production industries. With ongoing digitization, these systems are becoming increasingly indispensable. In general, the utilized technologies can be subdivided in data-driven and model-based approaches, whereas each category has its advantages and disadvantages. In the field of data-driven motion synthesis, recent works present deep learning based approaches for full body motion synthesis, which offer great potential for modeling natural motions, while considering heterogeneous influence factors. In this paper, we propose a novel deep blending approach for blending collision-free and feasible postures between a humanoid start and target posture. The network has been trained utilizing the CMU database to generate feasible postures. The proposed approach can be utilized for posture-blending, motion synthesis with known start and end-posture or key-frame animation. A preliminary evaluation indicates the validity and the potential of the novel approach.

Cognitive impairment such as memory loss, an impaired executive function and decreasing motivation can gradually undermine instrumental activities of daily living (IADL). With an older growing population, previous works have explored assistive technologies (ATs) to automate repetitive components of therapy and thereby increase patients' autonomy and reduce dependence on carers. While most ATs were built around screens and projection-based augmented reality (AR), the potential of head-mounted displays (HMDs) for therapeutic assistance is still under-explored. As a contribution to this effort we present cARe, an HMD-based AR framework that uses in-situ instructions and a guidance mechanism to assist patients with manual tasks. In a case study with six geriatric patients, we investigated the prototype's feasibility during a cooking task in comparison to a regular paper-based recipe. Qualitative and quantitative results indicate that cARe has potential to offer assistance to older individuals with declining cognitive function in their day-to-day tasks and increase their independence in an enjoyable way.

Cognitive impairment such as memory loss, an impaired executive function and decreasing motivation can gradually undermine instrumental activities of daily living (IADL). With an older growing population, previous works have explored assistive technologies (ATs) to automate repetitive components of therapy and thereby increase patients' autonomy and reduce dependence on carers. While most ATs were built around screens and projection-based augmented reality (AR), the potential of head-mounted displays (HMDs) for therapeutic assistance is still under-explored. In this interactive demonstration, we present cARe, an HMD-based AR framework that uses in-situ instructions and a guidance mechanism to assist patients with manual tasks. In a case study with six geriatric patients, we investigated the prototype's feasibility during a cooking task in comparison to a regular paper-based recipe. Qualitative and quantitative results indicate that cARe has potential to offer assistance to older individuals with declining cognitive function in their day-to-day tasks and increase their independence in an enjoyable way.

It is likely that a highly automated vehicle will be able to detect dangerous situations and also determine if the driver is reacting accordingly while driving manually. The car could then take over control to avoid imminent danger. A study was conducted to identify situations that are perceived as highly dangerous by drivers. To eliminate the general impression of having the situation under control, participants had to decide about system interventions for another driver in a self-programmed game. Results show that by using this approach, situations, like tailgating, can be identified that are perceived as dangerous by everybody. Additionally, collecting direct feedback about situations, where people would want their car to interfere with their own driving, is showing discrepancies to the results of the gamified approach. Using elements of gamification is a promising procedure for studies like this but must be considered during the analysis of the results.

Natural haptic feedback in virtual reality (VR) is complex andchallenging, due to the intricacy of necessary stimuli and re-spective hardware. Pseudo-haptic feedback aims at providinghaptic feedback without providing actual haptic stimuli butby using other sensory channels (e.g. visual cues) for feed-back. We combine such an approach with the additional inputmodality of muscle activity that is mapped to a virtual force toinfluence the interaction flow.

Abstract The simulation of two‐dimensional human locomotion in a bird's eye perspective is a key technology for various domains to realistically predict walk paths. The generated trajectories, however, are frequently deviating from reality due to the usage of simplifying assumptions. For instance, common deterministic motion planning algorithms predominantly utilize a set of static steering parameters (e.g. maximum acceleration or velocity of the agent) to simulate the walking behaviour of a person. This procedure neglects important influence factors, which have a significant impact on the spatio‐temporal characteristics of the finally resulting motion—such as the operator's physical conditions or the probabilistic nature of the human locomotor system. In overcome this drawback, this paper presents an approach to derive probabilistic motion models from a database of captured human motions. Although being initially designed for industrial purposes, this method can be applied to a wide range of use cases while considering an arbitrary number of dependencies (input) and steering parameters (output). To underline its applicability, a probabilistic steering parameter model is implemented, which models velocity, angular velocity and acceleration as a function of the travel distances, path curvature and height of a respective person. Finally, the technical performance and advantages of this model are demonstrated within an evaluation.

While the real world provides humans with a huge variety of sensory stimuli, virtual worlds most of all communicate their properties by visual and auditory feedback due to the design of current head mounted displays (HMDs). Since HMDs offer sufficient contact area to integrate additional actuators, prior works utilised a limited amount of haptic actuators to integrate respective information about the virtual world. With the Face/On prototype complex feedback patterns are introduced that combine a high number of vibration motors with additional thermal sources to transport multi-modal and spatial information. A pre-study determining the boundaries of the feedbacks' intensities as well as a user study showing a significant increase of presence and enjoyment validate Face/On's approach.

Current approaches for locomotion in virtual reality are either creating a visual-vestibular conflict, which is assumed to cause simulator sickness, or use metaphors such as teleportation to travel longer distances, lacking the perception of self motion. We propose VRSpinning, a seated locomotion approach based around stimulating the user's vestibular system using a rotational impulse to induce the perception of linear self-motion. In a first study we explored the approach of oscillating the chair in different frequencies during visual forward motion and collected user preferences on applying these feedback types. In a second user study we used short bursts of rotational acceleration to match the visual forward acceleration. We found that this rotational stimulus significantly reduced simulator sickness and increased the perception of self-motion in comparison to no physical motion.

Simulator studies have been conducted in the automotive domain since the 1960s. Recently, automated driving studies have become more popular as real-world automated cars start to emerge but at this time not all levels of automation can be realized. A simulation does not entail all details of real driving, creating a realistic simulation experience - both on a psychological and physical level - proposes recurring challenges. These are among others: sample acquisition, simulator sickness, simulator training, interface design, take-over requests and secondary tasks in automated driving simulator studies. In this paper, we review existing literature and summarize important lessons from simulations in the domain of driving automation to provide considerations for studies investigating driver behavior in the age of highly automated driving.

A function of an automated driving vehicle that can override a human driver while driving manually could work as a guardian angel in the car. It can take over control if it detects an imminent accident and has a possibility to avoid it. Because of the urgency of the intervention, there is not enough time to warn the driver in advance. In a study, feedback was collected from users how they perceived such an action while driving in a simulator. Additional feedback was collected about the general design and user interface of such a system. From an ethical point of view, we discovered discrepancies in the views of our participants regarding automated driving functions that need to be addressed in future development.

Mobile VR HMDs enable scenarios where they are being used in public, excluding all the people in the surrounding (Non-HMD Users) and reducing them to be sole bystanders. We present FaceDisplay, a modified VR HMD consisting of three touch sensitive displays and a depth camera attached to its back. People in the surrounding can perceive the virtual world through the displays and interact with the HMD user via touch or gestures. To further explore the design space of FaceDisplay, we implemented three applications (FruitSlicer, SpaceFace and Conductor) each presenting different sets of aspects of the asymmetric co-located interaction (e.g. gestures vs touch). We conducted an exploratory user study (n=16), observing pairs of people experiencing two of the applications and showing a high level of enjoyment and social interaction with and without an HMD. Based on the findings we derive design considerations for asymmetric co-located VR applications and argue that VR HMDs are currently designed having only the HMD user in mind but should also include Non-HMD Users.

Including haptic feedback in current consumer VR applications is frequently challenging, since technical possibilities to create haptic feedback in consumer-grade VR are limited. While most systems include and make use of the possibility to create tactile feedback through vibration, kinesthetic feedback systems almost exclusively rely on external mechanical hardware to induce actual sensations so far. In this paper, we describe an approach to create a feeling of such sensations by using unmodified off-the-shelf hardware and a software solution for a multi-modal pseudo-haptics approach. We first explore this design space by applying user-elicited methods, and afterwards evaluate our refined solution in a user study. The results show that it is indeed possible to communicate kinesthetic feedback by visual and tactile cues only and even induce its perception. While visual clipping was generally unappreciated, our approach led to significant increases of enjoyment and presence.

Virtual reality (VR) technology strives to enable a highly immersive experience for the user by including a wide variety of modalities (e.g. visuals, haptics). Current VR hardware however lacks a sufficient way of communicating the perception of weight of an object, resulting in scenarios where users can not distinguish between lifting a bowling ball or a feather. We propose a solely software based approach of simulating weight in VR by deliberately using perceivable tracking offsets. These tracking offsets nudge users to lift their arm higher and result in a visual and haptic perception of weight. We conducted two user studies showing that participants intuitively associated them with the sensation of weight and accept them as part of the virtual world. We further show that compared to no weight simulation, our approach led to significantly higher levels of presence, immersion and enjoyment. Finally, we report perceptional thresholds and offset boundaries as design guidelines for practitioners.

The form factors of current wearable devices are designed and limited to be worn at specifically defined on-body locations (such as the wrist), which can limit the interaction capabilities based on physical constraints in body movement and positioning. We investigate the design of a multi-functional wearable input device that can be worn at various locations on the body and may as well get mounted onto objects in the environment. This allows users to adjust the device's location to different affordances of varying situations and use cases. We present a SnapBand as such a multi-location touch input device that can be quickly snapped to different locations.

We present Movelet, a self-actuated bracelet that can move along the user's forearm to convey feedback via its movement and positioning. In contrast to other eyes-free modalities such as vibro-tactile feedback, that only works momentarily, Movelet is able to provide sustained feedback via its spatial position on the forearm, in addition to momentary feedback by movement. This allows to continuously inform the user about the changing state of information utilizing their haptic perception. In a user study using the Movelet prototype, we found that users can blindly estimate the device's position on the forearm with an average deviation of 1.20cm to the actual position and estimate the length of a movemement with an average deviation of 1.44cm. This shows the applicability of position-based feedback using haptic perception.

Redirected walking (RDW) allows virtual reality (VR) users to walk infinitely while staying inside a finite physical space through subtle shifts (gains) of the scene to redirect them back inside the volume. All prior approaches measure the feasibility of RDW techniques based on if the user perceives the manipulation, leading to rather small applicable gains. However, we treat RDW as an interaction technique and therefore use visually perceivable gains instead of using the perception of manipulation. We revisited prior experiments with focus on applied gains and additionally tested higher gains on the basis of applicability in a user study. We found that users accept curvature gains up to 20°/m, which reduces the necessary physical volume down to approximately 6x6m for virtually walking infinitely straight ahead. Our findings strife to rethink the usage of redirection from being unperceived to being applicable and natural.

Within the manufacturing industry, digital modelling activities and the simulation of human motion in particular, have emerged during the last decades. For the use-case of walk path planning, however, recent path planning approaches on the one hand reveal drawbacks in terms of realism and naturalness of motion. On the other hand, the generation of variant-rich travel routes by means of modeling the statistical nature of human motion has not been explored yet. In order to contribute to a better prediction quality of planning models, this paper therefore presents an approach for realistically simulating walk paths of single subjects. In order to take into consideration the variability of human locomotion, a statistical model describing human motion in a two-dimensional bird's eye view is presented. This model is generated from a comprehensive database (20 000 steps) of captured human motion and covers a wide range of gait variants. In order to obtain short and collision-free trajectories, this approach is combined with a path planning algorithm. The utilized hybrid A* path planner can be regarded as an orchestration-instance, stitching together succeeding left and right steps, which were drawn from the statistical motion model. Although being initially designed for industrial purposes, this method can be applied to a wide range of use-cases beyond automotive walk path planning. To underline the evident benefits of the proposed approach, the novel motion planner's technical performance is demonstrated in an evaluation.

Recently, the significant technological developments of head-mounted displays and tracking systems have boosted a widespread use of immersive virtual reality in the manufacturing industry. Regardless of the respective use-case, however, methods to ensure the validity of the human motion being performed in such virtual environments remain largely unaddressed. In the context of human locomotion, previous work present models quantifying behavioral differences between virtual reality and the real world. However, those findings are not used in control experiments to post-hoc counterbalance VR-induced performance modulations. Consequently, the prediction quality of previous analyses is not known. This paper bridges this gap, by testing such a behavior model in the context of an independent experiment (n = 10). The evaluation shows that a model derived from literature can indeed be used to post-hoc correct temporal disparities between locomotion in the real world and in virtual reality.

Continuous advances in personal audio technology (e.g. headphones), led to efficient noise cancellation and allowed users to build and influence their personal acoustic environment. Despite the high adoption and ubiquitous character of the technology, we do not fully understand which particular factors influence and form usage patterns. As a step towards understanding the usage of personal audio technology, we conducted two focus groups (n=10) to investigate current headphone usage and users' wishes regarding current and future personal audio technology. Based on this data, we derive a model for what we call personal soundscape curation. This model was assessed with the data of a crowdsourced survey on Amazon Mechanical Turk (n=194) on state of the art practices. Personal soundscape curation allows to describe usage strategies (curation, adaptation, renunciation) and break down influencing factors of context and environment as well as illustrate which consequences may arise from the users' behavior.

Most current virtual reality (VR) head-mounted displays (HMD) create a highly immersive experience and are currently becoming part of the living room entertainment (e.g. PSVR). However, current VR systems focus mainly on increasing the immersion and enjoyment for the user wearing the HMD (HMD user). This results in all the bystanders (Non-HMD users) in the living room being excluded from the experience and degraded to mainly observing the HMD user. In this demonstration we show ShareVR, a VR system using floor projection and mobile displays in combination with positional tracking to visualize the virtual world for the Non-HMD user, enabling them to interact with the HMD user and become part of the VR experience. Additionally, we implemented several experiences for the asymmetric nature of ShareVR, exploring its design space.

Mobile VR HMDs enable users to experience virtual reality content in a variety of nomadic scenarios, excluding all the people in the surrounding (Non-HMD Users) and reducing them to be sole bystanders. This leads to a scenario where the HMD User experiences a sense of isolation and the Non-HMD Users a sense of exclusion. To battle these phenomena we present FaceDisplay, a modified VR HMD consisting of three touch sensitive displays and a depth camera attached to its back. This allows Non-HMD User to see inside the immersed users virtual world and enable them to interact via touch and gestures. We built a VR HMD prototype consisting of three additional screens and present interaction techniques and an example application that leverage the FaceDisplay design space.

We perceive the flow of time as a constant factor in the real world, but there are examples in media, like films or games, where time is being manipulated and slowed down. Manipulating temporal cues is simple in linear media by slowing down video and audio. Interactive media like VR however poses additional challenges, because user interaction speed is independent from media speed. While the speed of the environment can still be manipulated easily, interaction is a new aspect to consider. We implemented such manipulation by slowing down visual feedback of user movements. In prior experiments we slowed down the virtual representation of a user by applying a velocity based low pass filter and by visually redirecting the motion. We found such a manipulation to be even contributing to realism, enjoyment or presence as long as it is consistent with the experience.

With the rise of immersive visualization techniques, many domains within the manufacturing industry are increasingly validating production processes in virtual reality (VR). The validity of the results gathered in such simulations, however, is widely unknown—in particular, with regard to human locomotion behavior. To bridge this gap, this article presents an experiment analyzing the behavioral disparity between human locomotion being performed without any equipment and in immersive VR while wearing a head-mounted display (HMD). The presented study (n= 30) is split up in three sections and covers linear walking, non-linear walking, and obstacle avoidance. Special care has been given to design the experiment so that findings are generally valid and can be applied to a wide range of domains beyond the manufacturing industry. The findings provide novel insights into the effect of immersive VR on specific gait parameters. In total, a comprehensive sample of 18.09km is analyzed. The results reveal that the HMD had a medium effect (up to 13%) on walking velocity, on non-linear walking toward an oriented target, and on clearance distance. The overall differences are modeled using multiple regression models, thus allowing the general usage within various domains. Summarizing, it can be concluded that VR can be used to analyze and plan human locomotion; however, specific details may have to be adjusted to transfer findings to the real world.

Virtual reality (VR) head-mounted displays (HMD) allow for a highly immersive experience and are currently becoming part of the living room entertainment. Current VR systems focus mainly on increasing the immersion and enjoyment for the user wearing the HMD (HMD user), resulting in all the bystanders (Non-HMD users) being excluded from the experience. We propose ShareVR, a proof-of-concept prototype using floor projection and mobile displays in combination with positional tracking to visualize the virtual world for the Non-HMD user, enabling them to interact with the HMD user and become part of the VR experience. We designed and implemented ShareVR based on the insights of an initial online survey (n=48) with early adopters of VR HMDs. We ran a user study (n=16) comparing ShareVRto a baseline condition showing how the interaction using ShareVR led to an increase of enjoyment, presence and social interaction. In a last step we implemented several experiences for ShareVR, exploring its design space and giving insights for designers of co-located asymmetric VR experiences.

Mobile virtual reality (VR) head-mounted displays (HMDs) allow users to experience highly immersive entertainment whilst being in a mobile scenario. Long commute times make casual gaming in public transports and cars a common occupation. However, VR HMDs can currently not be used in moving vehicles since the car's rotation affects the HMD's sensors and simulator sickness occurs when the visual and vestibular system are stimulated with incongruent information. We present CarVR, a solution to enable VR in moving vehicles by subtracting the car's rotation and mapping vehicular movements with the visual information. This allows the user to actually feel correct kinesthetic forces during the VR experience. In a user study (n = 21), we compared CarVR inside a moving vehicle with the baseline of using VR without vehicle movements. We show that the perceived kinesthetic forces caused by CarVR increase enjoyment and immersion significantly while simulator sickness is reduced compared to a stationary VR experience. Finally, we explore the design space of in-car VR entertainment applications using real kinesthetic forces and derive design considerations for practitioners.

The integration of multi-sensory stimuli, e.g. haptic airflow, in virtual reality (VR) has become an important topic of VR research and proved to enhance the feeling of presence. VaiR focuses on an accurate and realistic airflow simulation that goes far beyond wind. While previous works on the topic of airflow in VR are restricted to wind, while focusing on the feeling of presence, there is to the best of our knowledge no work considering the conceptual background or on the various application areas. Our pneumatic prototype emits short and long term flows with a minimum delay and is able to animate wind sources in 3D space around the user's head. To get insights on how airflow can be used in VR and how such a device should be designed, we arranged focus groups and discussed the topic. Based on the gathered knowledge, we developed a prototype which proved to increase presence, as well as enjoyment and realism, while not disturbing the VR experience.

Deaf individuals often experience communication difficulties in face-to-face interactions with hearing people. In order to support deaf individuals in such situations, an active stream of assistive technology (AT) research focuses on real-time translation of sign language. We investigate the impact of real-time translation-based ATs on communication quality between deaf and hearing individuals. We conducted a focus group and aWizard of Oz study in which deaf and hearing participants jointly interacted with different assistive technologies. We find that while ATs facilitate communication, communication quality is degraded by to breaks in the conversation. Using Co-Cultural Theory, we identify deaf people as a subordinate group inside a hearing society. Our results indicate that current ATs reinforce this subordination by emphasizing deficiency of mastering the dominant form of communication. Based on our findings, we propose a change in design perspective by enabling the hearing to sign rather than the deaf to "hear". We argue that ATs should not be seen as "just" a tool for the Deaf but rather as a collaborative technology.

While we perceive time as a constant factor in the real world, it can be manipulated in media. Being quite easy for linear media, this is used for various aspects of storytelling e.g., by applying slow motion in movies or TV. Interactive media like VR however poses additional challenges, because user interaction speed is independent from media speed. While it is still possible to change the speed of the environment, for interaction it is also necessary to deal with the emerging speed mismatch, e.g., by slowing down visual feedback of user movements. In this paper, we explore the possibility of such manipulations of visual cues, with the goal of enabling the use of slow motion also in immersive interactive media like VR. We conducted a user study to investigate the impact of limiting angular velocity of a virtual character in first person view in VR. Our findings show that it is possible to use slow motion in VR while maintaining the same levels of presence, enjoyment and susceptibility to motion sickness, while users adjust to the maximum speed quickly. Moreover, our results also show an impact of slowing down user movements on their time estimations.

We present SwiVRChair, a motorized swivel chair to nudge users' orientation in 360 degree storytelling scenarios. Since rotating a scene in virtual reality (VR) leads to simulator sickness, storytellers currently have no way of controlling users' attention. SwiVRChair allows creators of 360 degree VR movie content to be able to rotate or block users' movement to either show certain content or prevent users from seeing something. To enable this functionality, we modified a regular swivel chair using a 24V DC motor and an electromagnetic clutch. We developed two demo scenarios using both mechanisms (rotate and block) for the Samsung GearVR and conducted a user study (n=16) evaluating the presence, enjoyment and simulator sickness for participants using SwiVRChair compared to self control (Foot Control). Users rated the experience using SwiVRChair to be significantly more immersive and enjoyable whilst having a decrease in simulator sickness.

We present a gaze-based interaction approach that maps a single rotary control knob to multiple interfaces in the car using the user's gaze to determine the respective interaction context. By this, the complexity of physical interfaces (e.g. number of buttons) can be reduced and abstracted to a single remaining physical control interface located at the steering wheel to allow for interaction that doesn't require drivers to take their hands off the steering wheel. We implemented a prototype that uses multiple eye-trackers in a driving simulator to explore the concept of gaze mapping.

We present FaceTouch, a novel interaction concept for mobile Virtual Reality (VR) head-mounted displays (HMDs) that leverages the backside as a touch-sensitive surface. With FaceTouch, the user can point at and select virtual content inside their field-of-view by touching the corresponding location at the backside of the HMD utilizing their sense of proprioception. This allows for rich interaction (e.g. gestures) in mobile and nomadic scenarios without having to carry additional accessories (e.g. a gamepad). We built a prototype of FaceTouch and conducted two user studies. In the first study we measured the precision of FaceTouch in a display-fixed target selection task using three different selection techniques showing a low error rate of 2% indicate the viability for everyday usage. To asses the impact of different mounting positions on the user performance we conducted a second study. We compared three mounting positions of the touchpad (face, hand and side) showing that mounting the touchpad at the back of the HMD resulted in a significantly lower error rate, lower selection time and higher usability. Finally, we present interaction techniques and three example applications that explore the FaceTouch design space.

We present GyroVR, head worn flywheels designed to render inertia in Virtual Reality (VR. Motions such as flying, diving or floating in outer space generate kinesthetic forces onto our body which impede movement and are currently not represented in VR. We simulate those kinesthetic forces by attaching flywheels to the users head, leveraging the gyroscopic effect of resistance when changing the spinning axis of rotation. GyroVR is an ungrounded, wireless and self contained device allowing the user to freely move inside the virtual environment. The generic shape allows to attach it to different positions on the users body. We evaluated the impact of GyroVR onto different mounting positions on the head (back and front) in terms of immersion, enjoyment and simulator sickness. Our results show, that attaching GyroVR onto the users head (front of the Head Mounted Display (HMD)) resulted in the highest level of immersion and enjoyment and therefore can be built into future VR HMDs, enabling kinesthetic forces in VR.

Belt is a novel unobtrusive input device for wearable displays that incorporates a touch surface encircling the user's hip. The wide input space is leveraged for a horizontal spatial mapping of quickly accessible information and applications. We discuss social implications and interaction capabilities for unobtrusive touch input and present our hardware implementation and a set of applications that benefit from the quick access time. In a qualitative user study with 14 participants we found out that for short interactions (2-4 seconds), most of the surface area is considered as appropriate input space, while for longer interactions (up to 10 seconds), the front areas above the trouser pockets are preferred.

This paper presents Glass Unlock, a novel concept using smart glasses for smartphone unlocking, which is theoretically secure against smudge attacks, shoulder-surfing, and camera attacks. By introducing an additional temporary secret like the layout of digits that is only shown on the private near-eye display, attackers cannot make sense of the observed input on the almost empty phone screen. We report a user study with three alternative input methods and compare them to current state-of-the-art systems. Our findings show that Glass Unlock only moderately increases authentication times and that users favor the input method yielding the slowest input times as it avoids focus switches between displays.

In this paper we present ColorSnakes, a PIN-based authentication mechanism for smartphones which uses fake paths on a grid of numbers to disguise user input. In a lab study (n=24),we evaluated variations of ColorSnakes in terms of usability and security. In comparison to direct input, indirect input significantly reduced the risk of shoulder surfing (10.5%) without increasing the input time. In a follow up real-world study (n=12), we compared ColorSnakes with PIN entry and Android's Pattern Unlock over the course of three weeks. Although authentication time for ColorSnakes was higher than for the other two mechanisms, participants valued the security benefit over its slightly higher error rate and increased authentication time. We argue that ColorSnakes could be used as an additional authentication mechanism alongside current mechanisms, thus providing the user with the choice of changing to ColorSnakes for certain applications or when there is an observer.

This paper is the first to assess the impact of touchscreen damage on smartphone interaction. We gathered a dataset consisting of 95 closeup images of damaged smartphones and extensive information about a device's usage history, damage severity, and impact on use. 88% of our participants continued to use their damaged smartphone for at least three months; 32% plan to use it for another year or more, mainly due to high repair and replacement costs. From the dataset, we identified three categories of damaged smartphone displays. Reading and text input were most affected. Further interviews (n=11) revealed that users adapt to damage with diverse coping strategies, closely tailored to specific interaction issues. In total, we identified 23 different strategies. Based on our results, we proposed guidelines for interaction design in order to provide a positive user experience when display damage occurs.

To maintain a mobile form factor, the screen real estate of a mobile device canIn this paper we present SurfacePhone; a novel configuration of a projector phone which aligns the projector to project onto a physical surface to allow tabletop-like interaction in a mobile setup. The projection is created behind the upright standing phone and is touch and gesture-enabled. Multiple projections can be merged to create shared spaces for multi-user collaboration. We investigate this new setup, starting with the concept that we evaluated with a concept prototype. Furthermore we present our technical prototype, a mobile phone case with integrated projector that allows for the aforementioned interaction. We discuss its technical requirements and evaluate the accuracy of interaction in a second user study. We conclude with lessons learned and design guidelines.

The vision of pervasive ambient information displays which show relevant information has not yet come true. One of the main reasons is the limited number of available displays in the environment which is a fundamental requirement of the original vision. We introduce the concept of an Ambient Mobile Pervasive Display AMP-D which is a wearable projector system that constantly projects an ambient information display in front of the user. The floor display provides serendipitous access to public and personal information. The display is combined with a projected display on the user's hand, forming a continuous interaction space that is controlled by hand gestures. The paper introduces this novel device concept, discusses its interaction design, and explores its advantages through various implemented application examples. Furthermore, we present the AMP-D prototype which illustrates the involved challenges concerning hardware, sensing, and visualization.

Handheld displays enable flexible spatial exploration of information spaces -- users can physically navigate through three-dimensional space to access information at specific locations. Having users constantly hold the display, however, has several limitations: (1) inaccuracies due to natural hand tremors; (2) fatigue over time; and (3) limited exploration within arm's reach. We investigate autonomous, self-actuated displays that can freely move and hold their position and orientation in space without users having to hold them at all times. We illustrate various stages of such a display's autonomy ranging from manual to fully autonomous, which -- depending on the tasks -- facilitate the interaction. Further, we discuss possible motion control mechanisms for these displays and present several interaction techniques enabled by such displays. Our Hover Pad toolkit enables exploring five degrees of freedom of self-actuated and autonomous displays and the developed control and interaction techniques. We illustrate the utility of our toolkit with five prototype applications, such as a volumetric medical data explorer.

We present a novel approach to graspable interfaces using Touchbugs, actuated physical objects for interacting with interactive surface computing applications. Touchbugs are active tangibles that are able to move across surfaces by employing vibrating motors and can communicate with camera based multi-touch surfaces using infrared LEDs. Touchbug's embedded inertial sensors and computational capabilities open a new interaction space by providing autonomous capabilities for tangibles that allow goal directed behavior.

Current interactive surfaces do not support user identification. Hence, personalized applications that consider user-specific access control are not possible. Diverse approaches for identifying and distinguishing users have been investigated in previous research. Token-based approaches -- e.g., which utilize the user's mobile phone -- are especially promising, as they also allow for consideration of the user's personal digital context (e.g., stored messages, contacts, or media data). However, existing interaction techniques are limited regarding their ability to enable users to manipulate (e.g., select or copy) multiple items at the same time, as they are cumbersome when the number of files exceeds a certain amount. We present MobiZone, a technique that enables users to interact with large numbers of items on an interactive surface while enabling personalized access by using the mobile phone as a token. MobiZone provides a spatial zone that can be positioned, resized and associated with any action according to the user's needs; items enclosed by the zone can be manipulated simultaneously. We present three interaction techniques (FlashLight&Control, Remote&Control, and Place&Control) that enable users to control the zone. Additionally, we report the results of a comparative user study in which we compared the different interaction techniques for MobiZone. The results indicate that users are fastest with Remote&Control, and they also rated Remote&Control slightly higher than the other techniques.

Many interactions naturally extend across smart-phones and devices with larger screens. Indeed, data might be received on the mobile but more conveniently processed with an application on a larger device, or vice versa. Such interactions require spontaneous data transfer from a source location on one screen to a target location on the other device. We introduce a cross-device Drag-and-Drop technique to facilitate these interactions involving multiple touchscreen devices, with minimal effort for the user. The technique is a two-handed gesture, where one hand is used to suitably align the mobile phone with the larger screen, while the other is used to select and drag an object between devices and choose which application should receive the data.

We present the technical framework supporting a cross-device Drag-and-Drop technique designed to facilitate interactions involving multiple touchscreen devices. This technique supports users that need to transfer information received or produced on one device to another device which might be more suited to process it. Furthermore, it does not require any additional instrumentation. The technique is a two-handed gesture where one hand is used to suitably align the mobile phone with the larger screen, while the other is used to select and drag an object from one device to the other where it can be applied directly onto a target application. We describe the implementation of the framework that enables spontaneous data-transfer between a mobile device and a desktop computer.

Natural forms of interaction have evolved for personal devices that we carry with us (mobiles) as well as for shared interactive displays around us (surfaces) but interaction across the two remains cumbersome in practice. We propose a novel cross-device interaction style for mobiles and surfaces that uses the mobile for tangible input on the surface in a stylus-like fashion. Building on the direct manipulation that we can perform on either device, it facilitates fluid and seamless interaction spanning across device boundaries. We provide a characterization of the combined interaction style in terms of input, output, and contextual attributes, and demonstrate its versatility by implementation of a range of novel interaction techniques for mobile devices on interactive surfaces.

With the rise of current smartphones, virtual keyboards for touchscreens became the dominant mobile text entry technique. We developed a typing game that records how users touch on the standard Android keyboard to investigate users' typing behaviour. 47,770,625 keystrokes from 72,945 installations have been collected by publishing the game. By visualizing the touch distribution we identified a systematic skew and derived a function that compensates this skew by shifting touch events. By updating the game we conduct an experiment that investigates the effect of shifting touch events, changing the keys' labels, and visualizing the touched position. Results based on 6,603,659 keystrokes and 13,013 installations show that visualizing the touched positions using a simple dot decreases the error rate of the Android keyboard by 18.3% but also decreases the speed by 5.2% with no positive effect on learnability. The Android keyboard outperforms the control condition but the constructed shift function further improves the performance by 2.2% and decreases the error rate by 9.1%. We argue that the shift function can improve existing keyboards at no costs.

Recent mobile phones allow users to perform a multitude of different tasks. Complexity of tasks challenges the design of mobile applications in many ways. For instance, the limited screen space of mobile devices allows only a small number of items to be displayed. Therefore, users often have to change the view or have to resize the displayed content (e.g., images or maps) to view the required information. We present the system MobIES, which allows users to extend the mobile interfaces of their mobiles phones using external screens. Users connect their mobile phone with an external display by holding it on the border of the external display. When the connection is established, the user interface of the currently active mobile application is distributed on the phone and the external screen. This enables users to take advantage of using existing screens in their environments and temporarily benefit from an extended screen space. In this paper, we discuss the concept of MobIES and present a prototype implementation.

Public terminals for service provision provide high convenience to users due to their constant availability. Yet, the interaction with them lacks security and privacy as it takes place in a public setting. Additionally, users have to wait in line until they can interact with the terminal. In comparison to that, personal mobile devices allow for private service execution. Since many services, like with-drawing money from an ATM, require physical presence at the terminal, hybrid approaches have been developed. These move parts of the interaction to a mobile device. In this work we present the results of a four week long real world user study, in which we investigated whether hybrid approaches would actually be used. The results show that users accept the hybrid service as they understood that they could use down downtimes (like bus rides) to prepare the interaction with the public terminal. Our findings give novel insights about security relevant aspects such as where and when users interact with the mobile service before accessing the public terminal. So the preparation of the transaction on the mobile phone was often conducted much further away from the terminal than expected (81.0% with a distance greater than 400m) and earlier than expected (82.1% at least 5 minutes in advance).

Sharing social and contextual information via services like Facebook, Twitter or Foursquare has become extremely popular in the recent years. This paper introduces the novel MyState concept in which users can augment any kind of object with Near Field Communication (NFC) tags, can write any social or contextual information on those tags using their mobile phones and can publish this information on a social networking site just by touching such a tag with their phone. The distinct features of MyState are A) the possibility to augment any personal or public object with any contextual or social information, B) the possibility that everybody can touch those tags in order to post the related information to a social networking site, C) the speed and convenience to publish information by a simple touch as users don't have to look at the mobile phone screen, interact with mobile phone menus or write any text when touching an already deployed tag. The paper reports on two field studies which provide insights on where the participants placed the tags, how they used MyState and what type of information was shared. Here we observed that users typically shared identity, location, activity and time, but also feelings, social meanings and experiences. Furthermore we identified several distinct social usage patterns such as synchronizing activities, expressing moods, games and tracking shared items.

Touchscreens became the dominant input device for smartphones. Users' touch behaviour has been widely studied in lab studies with a relative low number of participants. In contrast, we published a game in the Android Market that records the touch behaviour when executing a controlled task to collect large amounts of touch events. Players' task is to simply touch circles appearing on the screen. Data from 91,731 installations has been collected and players produced 120,626,225 touch events. We determined the error rates for different target sizes and screen locations. The amount of data enabled us to show that touch positions are systematically skewed. A compensation function that shifts the users' touches to reduce the amount of errors is derived from the data and evaluated by publishing an update of the game. The independent-measures experiment with data from 12,201 installations and 15,326,444 touch events shows that the function reduces the error rate by 7.79%. We argue that such a compensation function could improve the touch performance of virtually every smartphone user.

Currently, the development of mobile applications heavily relies on using conventional computers as development platform. MobiDev enables people in emerging countries without access to a computer but to a cell phone to develop their own locally relevant applications. The goal of the Mo-biDev project is to simplify development and deployment of applications directly on mobile phones. As a first step, we focus on the design of applications and try to support the computer science curriculum in developing countries to bootstrap the mobile developer culture and community. MobiDev allows the creation of graphical user interfaces (GUI) using various concepts. We present the results of a first system evaluation that show how people perceive the concepts for UI creation of MobiDev.

Today, one does not have to travel far to find examples of digital signage, yet the adoption of interactive gesture based public displays remains quite rare. Subsequently, not much is known about them despite a large array of potential advantages. This paper contributes to our understanding of how people perceive, respond to and interact with such displays outside the controlled environment of a research lab. Unlike other works which have focused on isolated aspects of in-lab interaction, we present a detailed examination that addresses a wide range of responses to such a display - including those who ignore them completely. To facilitate our study we created an experimental coarse gesture based software suite and then deployed the system along with associated applications as part of an existing large scale public display network. Using this as a base, we executed four studies designed to passively observe the reactions of passers-by and followed these up with a fifth controlled experiment that compared the effectiveness of two different kinds of gesture in the context of menu item selection. To conclude, we present our key-findings and highlight possible avenues of further study for the future of gesture based digital signage.

This paper documents MultiKit: a toolkit that supports the development of sophisticated user-interfaces for applications using Near Field Communication (NFC). The usage of the toolkit is exemplified using three implemented prototypes, each with significant differences. These differences highlight the complexities involved in producing such NFC applications, and the benefits of a toolkit that can adapt to varying interface types. As the toolkit is tailored to NFC technology, NFC-specific features are explored. These include switching between dynamic (e.g. projection) and static interfaces (e.g. posters), and interface generation from NFC tag descriptions.

We introduce NaviRadar: an interaction technique for mobile phones that uses a radar metaphor in order to communicate the user's correct direction for crossings along a desired route. A radar sweep rotates clockwise and tactile feedback is provided where each sweep distinctly conveys the user's current direction and the direction in which the user must travel. In a first study, we evaluated the overall concept and tested five different tactile patterns to communicate the two different directions via a single tactor. The results show that people are able to easily understand the NaviRadar concept and can identify the correct direction with a mean deviation of 37° out of the full 360° provided. A second study shows that NaviRadar achieves similar results in terms of perceived usability and navigation performance when compared with spoken instructions. By using only tactile feedback, NaviRadar provides distinct advantages over current systems. In particular, no visual attention is required to navigate; thus, it can be spent on providing greater awareness of one's surroundings. Moreover, the lack of audio attention enables it to be used in noisy environments or this attention can be better spent on talking with others during navigation.

This paper reports on a development framework, two prototypes, and a comparative study in the area of multi-tag Near-Field Communication (NFC) interaction. By combining NFC with static and dynamic displays, such as posters and projections, services are made more visible and allow users to interact with them easily by interacting directly with the display with their phone. In this paper, we explore such interactions, in particular, the combination of the phone display and large NFC displays. We also compare static displays and dynamic displays, and present a list of deciding factors for a particular deployment situation. We discuss one prototype for each display type and developed a corresponding framework which can be used to accelerate the development of such prototypes whilst supporting a high level of versatility. The findings of a controlled comparative study indicate, among other things, that all participants preferred the dynamic display, although the static display has advantages, e.g. with respect to privacy and portability.

One shortcoming of self-describing smart objects augmented with digital resources is the limitation of output modalities due to their long established physical appearances. To overcome this drawback intangible representations e.g., sound, video projection etc. are usually coupled with the tangible representations of smart objects that enable access and interaction with their value added features. In this paper, we explore two mobile interaction techniques that associate such intangible representation to smart objects using a pico projector augmented camera phone. The first technique utilizes a Magic Lens metaphor applying mobile augmented reality (contextual information is overlaid while looking at a smart object through camera) to uncover and interact with smart objects. The second technique, Personal Projection follows similar mechanisms in discovery and interaction, except information is projected onto the nearest surface. We report the implementation of these two techniques and a comparative qualitative study with three prototype smart object applications. The findings give us deeper insights on the positive and negative aspects of these two techniques and open up a range of stimulating research issues that we discuss in the paper.

More and more people interact with their mobile phone while walking. The presented research analyzes; firstly, the negative effect of walking when considering reading and target selection tasks, such as weaker performance and higher workload. Here, we focused on one-handed interaction with a touch screen whereby the thumb is used as the input device. Secondly, we analyze how these negative effects can be compensated by increasing the text size and the size of the targets to select on the mobile phone. A comparative user study was conducted with 16 participants who performed target acquisition and reading tasks while standing and walking. The results show that whilst performance decreases, cognitive load increases significantly when reading and selecting targets when walking. Furthermore, the results show that the negative effect regarding target selection can be compensated by increasing the target size, but the text reading task did not yield better performance results for a larger text size due to the increased demand for scrolling. These results can be used to inform future designs of mobile user interfaces which might provide a dedicated walking mode.

PhoneTouch is a novel technique for integration of mobile phones and interactive surfaces. The technique enables use of phones to select targets on the surface by direct touch, facilitating for instance pick&drop-style transfer of objects between phone and surface. The technique is based on separate detection of phone touch events by the surface, which determines location of the touch, and by the phone, which contributes device identity. The device-level observations are merged based on correlation in time. We describe a proof-of-concept implementation of the technique, using vision for touch detection on the surface (including discrimination of finger versus phone touch) and acceleration features for detection by the phone.

Important drawbacks of map-based navigation applications for mobile phones are their small screen size and that users have to associate the information provided by the mobile phone with the real word. Therefore, we designed, implemented and evaluated the Rotating Compass - a novel public display for pedestrian navigation. Here, a floor display continuously shows different directions (in a clockwise order) and the mobile phone informs the user when their desired direction is indicated. To inform the user, the mobile phone vibrates in synchronization with the indicated direction. We report an outdoor study that compares a conventional paper map, a navigation application running on a mobile device, navigation information provided by a public display, and the Rotating Compass. The results provide clear evidence of the advantages of the new interaction technique when considering task completion time, context switches, disorientation events, usability satisfaction, workload and multi-user support.

Viewing and sharing media on mobile devices are popular scenarios, however, the limited screen size results in multiple users having to gather around a single display leading to an undesirable viewing experience. Similarly, sharing a single image with a single person is cumbersome and requires several steps to complete. This demonstration paper presents an approach in which a mobile phone is coupled with a personal projector to overcome the output limitations of mobile phones. The large mobile projected display allows several users to simultaneously view media. We also present a group based sharing approach allowing users to intuitively and easily share media between mobile devices. We reverse the typical role of sharing and introduce a technique whereby the receiver of the media is responsible for performing the sharing interaction. This allows sharing with a single member or multiple members of the group. Sharing with every member of the group is also easily possible.

An increasing number of households are equipped with a large number of TV sets and more and more of them are large highresolution displays. Furthermore, we see the integration of web browsing and email functionalities in these devices, which are then often controlled via a wireless mouse and keyboard. The latter were rather designed for the usage on a desk, rather then by a person sitting on their sofa in a living room. Therefore, this paper investigates the usage of a PDA, as a replacement which can be used for controlling a remote cursor and for text input. The results of the experimental comparison of these input devices show, as expected, the superiority of mouse and keyboard (as the study participants were very experienced with them). Surprising results were the task completion time and usability satisfaction when using the mobile device. These results show the applicability of using a mobile device for controlling an application on a remote screen. Using a mobile device provides the advantages that every person can e.g. use their own mobile phone or that these devices can be used in multi-user scenarios.

Exchanging data between a mobile phone and a computer such as a laptop is still a very cumbersome process. This paper presents two different techniques, touch & connect and touch & select, designed help to overcome this problem and facilitate and speed up spontaneous interactions between such devices. Using touch & connect, the user can physically touch a computer in order to pair a Bluetooth connection and initiate a file transfer between these two devices. Touch & select extends this concept in that users can select a specific object or location on the computer screen by simply touching it with the mobile phone. We report the implementation of these interaction techniques based on Near Field Communication (NFC) tags and present a formal, comparative study focusing on transferring images. The results provide clear evidence of the advantages of touch & connect and touch & select when compared with current Bluetooth-based implementations. Considering task completion time for uploading and downloading pictures, touch & select was 43% and touch & connect 31% faster than the conventional Bluetooth-based approach.

Uploading and downloading pictures between a mobile phone and a computer is still a very cumbersome process. Because of this, many users actually do not copy, move or backup their pictures onto another computer until the storage capacity of the mobile phone is reached. This paper presents the prototypes (and respective implementation details) of the touch & connect and touch & select interaction techniques. Both techniques allow the quick and easy exchange of pictures by touching the computer with the mobile device. The first interaction technique: touch & connect, allows the user to touch a computer with their mobile phone in order to establish a Bluetooth connection and initiate a file transfer between the two devices. The second interaction technique: touch & select, extends this concept and allows the selection of a specific picture or location on the computer screen by touching it with the mobile phone. The interaction techniques were implemented using Near Field Communication (NFC) tags attached to the computer and an NFC phone capable of reading those tags.

The workshop on Mobile Interaction with the Real World (MIRW 2009) will invite papers which focus on new mobile and wearable input and output interfaces which allow simpler and straightforward interactions with mobile services and applications. An inherit problem of current mobile devices are their limited output and input capabilities. This workshop continues a successful series of workshops (2006--2008) that focus on new approaches to overcome these issues. Examples are the usage of external visual interfaces (e.g. projector phones, public displays, interactive surfaces) and additional input capabilities (e.g. gestures, on-body interfaces, pointing) and innovative feedback mechanisms (e.g. tactile feedback). The workshop combines technical presentations with the presentation of prototypes and focused discussions to drive interaction between participants.

Use of spatial information to support discovery of interaction opportunities has been widely demonstrated. In this paper, we focus on the use of spatial interfaces for identification and selection of devices a mobile user encounters in their immediate environment. We contribute an experimental evaluation of two spatial interface conditions in comparison with a non-spatial condition. The two spatial interface conditions are a device list ordered by distance and an iconic map of devices as seen from the user's perspective and the non-spatial condition is an alphabetical list. Our results show an overall user preference for the iconic map over the spatial and alphabetical list. However, there was no clear preference for the spatial interfaces over the non-spatial condition with respect to user satisfaction and mental load.

It is expected that projector phones (mobile phones with built-in pico projectors) will hit the market by 2010. Such phones provide a completely new way to display information and interaction techniques. The system presented in this paper allows the simulation of these projector phones as the real devices are not yet available. Through this, we demonstrate that it is currently possible to design, develop and evaluate applications for projector phones. The system supports three different modalities in order to compare when which display (phone display, projection) should be used. This prototype system was used for the implementation of two applications in order to test the advantages and disadvantages of projector phones for two common scenarios, picture browsing and map interaction. This demonstration paper describes the hardware used for the simulation of projector phones and the two developed prototypes. These prototypes were used for two different studies discussed in [1] and [2].

It is expected that projector phones (mobile phones with an integrated pico projector) will hit the market in the next few years. So far no research exists regarding how mobile applications should be designed when using a projection and which applications will profit from such a large high-resolution display. This paper presents a comparative evaluation of picture browsing using projector phones. In a study we compared three different interaction techniques: phone display only, projection only and a combination of both. The results show that users prefer the projection-based interaction techniques and the large high-resolution projection leads to a reduced number of enlarge interactions. However, the task completion time results illustrate how familiar users are in using conventional phones with small screen size. The paper presents a comprehensive discussion of findings from our study and defines important guidelines which we consider as critical for the success of applications for projector phones in the future.

First working prototypes of mobile phones with integrated pico projectors have already been demonstrated and it is expected that such projector phones will be sold within the next three years. Applications that require interaction with large amounts of information will benefit from the large projection and its high resolution. This paper analyses the advantages and disadvantages of an integrated projector when interacting with maps, and discusses findings useful for the development of mobile applications for projector phones. We report in particular the implementation of an application that uses either the screen of the mobile phone, the projection or a combination of both. These three options were compared in a user study in which the participants had to perform three different tasks with each option. The results provide clear evidence for the positive aspects of using a built-in projector, but also show some negative aspects related to text input.

The limited screen size and resolution of current mobile devices can still be problematic for map, multimedia and browsing applications. In this paper we present Touch & Interact: an interaction technique in which a mobile phone is able to touch a display, at any position, to perform selections. Through the combination of the output capabilities of the mobile phone and display, applications can share the entire display space. Moreover, there is potential to realize new interaction techniques between the phone and display. For example, select & pick and select & drop are interactions whereby entities can be picked up onto the phone or dropped onto the display. We report the implementation of Touch & Interact, its usage for a tourist guide application and experimental comparison. The latter shows that the performance of Touch & Interact is comparable to approaches based on a touch screen; it also shows the advantages of our system regarding ease of use, intuitiveness and enjoyment.

Touch & Interact is an interaction technique which combines mobile phones and public displays. The motivation for the project is to overcome the intrinsic output limitations of mobile phones. Touch & Interact extends the phone output to a public display allowing both screens to share the display space. This concept is especially useful for separating public and private data -- only showing private data on the phone display. During interaction, the phone is used as a stylus for the public display and can touch the display at any location. The phone also lends additional capabilities to interactions by providing storage capabilities, additional feedback (e.g. audio and haptic) and input modalities (e.g. keypad and joystick). The underlying technology supporting Touch & Interact is Near Field Communication (NFC); the dynamic display includes a mesh of NFC tags which can be interacted with using an NFC phone. A tourist map application was used to explore and test Touch & Interact in an environment with rich functionality.

The Mobile HCI community is moving beyond the interaction between a single user and her mobile device taking the users environment into account. Mobile interaction with the real world concentrates on using mobile devices as tools to interact with real world objects. This workshop continues the successful mobile interaction with the real world workshops 2006 and 2007. Relevant topics include (but are not limited to) mobile interaction with the real world; mobile devices as user interfaces for terminals; and Frameworks, middleware and APIs for the development of applications that take mobile interactions with the real world into account. The workshop combines technical presentations with the presentation of prototypes and focused discussions to drive interaction between participants.

In our everyday life we often see objects or persons and are aware that there are related digital services such as an online ticket service when seeing a poster advertising a concert. Currently it is a rather time consuming activity to find the related information. Using our Contextual Bookmark system a user can define a snapshot with her mobile phone consisting of a picture, time stamp and location. Such a bookmark can then be stored on the mobile phone, exchanged with friends and in particular be used to access related videos, web pages, and other services. This helps the user to bridge the gap between the virtual and the real world in order to use related services. Combining content and context analysis objects are recognized without any visual markers or electronic tags. We would like to demonstrate our system based on a nomadic usage scenario in which a person defines a Contextual Bookmark of a movie trailer, buys the corresponding movie, plays the movie on a TV, and exchanges the bookmark with a friend.

The last few years have seen the emergence of two parallel trends: the first of such trends is set by technologies such as Near Field Communication, 2D Bar codes and RFID that support the association of digital information with virtually every object. By using these technologies, ordinary objects such as coffee mugs or advertisement posters provide digital information that can be easily processed. The second trend is set by (semantic) Web services that provide a way to automatically invoke functionalities across the Internet lowering interoperability barriers. The PERCI system, discussed in the article, provides a bridge between these two technologies allowing the invocation of Web services using the information gathered from the tags to effectively transform every object in a service proxy.

The proposed demonstration is based on the work performed in the PERCI project and presents a generic framework to access and interact with Web Services through mobile interaction with real world objects. The demonstration will put a focus on the front-end of the framework that comprises augmented posters for mobile ticketing as well as a mobile client application for the interaction with associated services. By supporting Physical Mobile Interaction techniques such as Touching or Pointing, the framework tries to make mobile service interaction more intuitive and shift its focus from the menus of mobile devices to augmented physical objects.

The proposed demonstration offers the possibility to access and interact with (web) services through Physical Mobile Interaction (PMI) with tag-based dynamic physical objects such as papers. Our demonstration supports several direct manipulation gestures in the real world and deals with physical objects able to provide system feedback to the user during the course of the interaction.

Although mobile services can be used ubiquitously, their employment and the interaction with them are still restricted by the constraints of mobile devices. In order to facilitate and leverage mobile interaction with services, we present a generic framework that combines semantic Web service technology and physical mobile interaction. This interaction paradigm uses mobile devices to extract information from augmented physical objects and use it for a more intuitive and convenient invocation of associated services. For that purpose, the presented framework exploits Web service descriptions for the automatic and dynamic generation of customizable user interfaces that support and facilitate physical mobile interaction. This generic approach to mobile interaction with services through the interaction with physical objects promises to meet the complementary development of the Internet of things. A user study with a prototype application for mobile ticketing confirms our concept and shows its limits

Context-aware mobile applications and systems have been extensively explored in the last decade and in the last few years we already saw promising products on the market. Most of these applications assume that context data is highly accurate. But in practice this information is often unreliable, especially when gathered from sensors or external sources. Previous research has argued that the system usability can be improved by displaying the uncertainty to the user. The research presented in this paper shows that it is not always an advantage to show the confidence of the context-aware application to the user. We developed a system for automatic form filling on mobile devices which fills in any web form with user data stored on the mobile device. The used algorithm generates rules which indicate with which probability which input field of a form should be filled in with which value. Based on this we developed two versions of our system. One shows the uncertainty of the system and one not. We then conducted a user study which shows that the user needs slightly more time and produces slightly more errors when the confidence of the system is visualized.

The main goal of the workshop is to discuss approaches that use a mobile device (e.g. mobile phone, smartphone, PDA) for interactions with objects in the real world. Relevant topics include (but are not limited to) mobile interaction with the real world; mobile devices as user interfaces for terminals and vending machines; and Frameworks, middleware and APIs for the development of applications that take mobile interactions with the real world into account. The workshop combines technical presentations with the presentation of prototypes and focussed discussions to drive interaction between participants.

No abstract available.