Haptic sensations in pen-based surface interactions within virtual reality (VR) rely on specialised haptic devices like pens or tablets. However, as VR becomes more mobile, reducing the need for external hardware is crucial, shifting the focus to head-mounted displays (HMDs) for haptic feedback. In this work, we explore head-mounted relocation of multisensory feedback, combining vibrations and audio, to simulate varying levels of surface roughness during pen-based surface interaction in VR. In a within-subject user study (N=18), we evaluated the influence of the combined technique compared to the vibration-only and audio-only techniques on the perceived surface roughness. Our results indicate that all relocation techniques allowed participants to distinguish at least three out of five roughness levels through their haptic and/or auditory senses. Moreover, while the combination of vibration and audio feedback produced the highest perceived levels of surface roughness for rougher simulated surfaces, the vibration-only approach enabled the perception of more distinguishable roughness levels in the lower range of simulated surfaces. Our findings provide early evidence that haptic surface sensations can be relocated via head-mounted simulation during pen-based interactions, offering initial insights for future HMDs that integrate multisensory feedback.

Fidelity describes how closely a replication resembles the original. It can be helpful to analyze how faithful interactions in virtual reality (VR) are to a reference interaction. In prior research, fidelity has been restricted to the simulation of reality—also called realism. Our definition includes other reference interactions, such as superpowers or fiction. Interaction fidelity is a multilayered concept. Unfortunately, different aspects of fidelity have either not been distinguished in scientific discourse or referred to with inconsistent terminology. Therefore, we present the Interaction Fidelity Model (IntFi Model). Based on the human-computer interaction loop, it systematically covers all stages of VR interactions. The conceptual model establishes a clear structure and precise definitions of eight distinct components. As a communication tool, it helps teams to understand and discuss fidelity in VR. It was reviewed through workshops with fourteen VR experts. We provide guidelines, diverse examples, and educational material to apply the IntFi Model universally to any VR experience and propose foundational research opportunities.

Despite irreversibility being omnipresent in the lifeworld, research on interactions making use of irreversibility in computing systems is still in the early stages. User freedom – provided by the undo functionality – is considered to be a pillar of “usable” computer systems, overcoming irreversibility. Within this paper, we set up a thought experiment, challenging the “undo feature” and instead take advantage of irreversibility in the interaction with physical computing systems (tangibles, robots, etc). First, we present three material speculations, each inherently utilizing irreversibility. Second, we elaborate on the concept of irreversible interactions by contextualizing our work with critical HCI discourses and deducing three design strategies. Finally, we discuss irreversibility as a design element for self-reflection, meaningful acting, and a sustainable relationship with technology. While previously individual aspects of irreversibility have been explored, we contribute a comprehensive discussion of irreversible interactions in HCI presenting artifacts, a conceptualization, design strategies, and application purposes.

Providing haptic feedback in virtual reality to make the experience more realistic has become a strong focus of research in recent years. The resulting haptic feedback systems differ greatly in their technologies, feedback possibilities, and overall realism making it challenging to compare different systems. We propose the Haptic Fidelity Framework providing the means to describe, understand and compare haptic feedback systems. The framework locates a system in the spectrum of providing realistic or abstract haptic feedback using the Haptic Fidelity dimension. It comprises 14 criteria that either describe foundational or limiting factors. A second Versatility dimension captures the current trade-off between highly realistic but application-specific and more abstract but widely applicable feedback. To validate the framework, we compared the Haptic Fidelity score to the perceived feedback realism of evaluations from 38 papers and found a strong correlation suggesting the framework accurately describes the realism of haptic feedback.

The use of voice assistants (VAs) is spreading widely. Most common VAs consist of a single, usually female voice that responds to the user’s inquiry. We designed a VA system appearing as a group of agents, each with a different voice and a specialized task domain. We conducted a quantitative user study comparing our multi-agent approach with a conventional single-agent assistant in a smart home scenario as virtual reality (VR) simulation. The results show significantly higher user experience ratings for the multi-agent concept. Based on our findings, we discuss the potentials and challenges of designing multi-party VA systems.

The urban realm is currently undergoing a transformation in which cities are being laced with sensors and networks of ever-connected devices. At the same time, more and more novel and advanced materials are finding their way into interaction design and Human-Computer Interaction (HCI) research, offering us new interaction possibilities. In this half-day online studio, attendees will have the opportunity to exchange and reflect on urban interaction designs for user engagement based on a set of novel, unconventional, or omnipresent materials in urban environments. We will further collect historical, public, and community locations of different social and societal meaning. Participants will be asked to develop hands-on concepts and use cases for the selected locations through a material-centered approach and think about new levels of user-environment engagement to extend and explore the current design space. The formulated concepts and use cases will be recorded in an online collaboration tool with the prospect to publish them as enhances for the new generation of media architecture.

Informed by modern imaging techniques, current medical navigation systems support physicians during a variety of interventions, such as needle-based operations. During these, an abundance of information is often displayed on monitors placed in positions that are uncomfortable for the operator to view. In this article, we address these issues with the concept and prototype of a customized flexible display that is placed on the patient’s body to provide the essential information at just the right location. We present an empirical evaluation comparing the flexible display against a control condition using a standard interventional monitor setup and an additional condition that combines both. Our results show that the flexible display significantly reduces task load while improving overall usability. Furthermore, we found indications that the flexible display reduces task completion time while also observing a negative effect on accuracy, which needs to be balanced carefully.

Professionals in domains like film, theater, or architecture often rely on physical models to visualize spaces. With virtual reality (VR) new tools are available providing immersive experiences with correct perceptions of depth and scale. However, these lack the tangibility of physical models. Using tangible objects in VR can close this gap but creates the challenges of producing suitable objects and interacting with them with only the virtual objects visible. This work addresses these challenges by evaluating tangibles with three haptic fidelities: equal disc-shaped tangibles for all virtual objects, Lego-built tangibles, and 3D-printed tangibles resembling the virtual shapes. We present results from a comparative study on immersion, performance, and intuitive interaction and interviews with domain experts. The results show that 3D-printed objects perform best, but Lego offers a good trade-off between fast creation of tangibles and sufficient fidelity. The experts rate our approach as useful and would use all three versions.

Needle-guiding templates are used for a variety of minimally invasive medical interventions. While physically supporting needle placement with a grid of holes, they lack integrated information where needles need to be inserted. Physicians must manually determine the correct holes based on the output of planning software - a workflow that is error-prone and lengthy. We address these issues by embedding a display into the template using electroluminescence (EL) screen printing. The EL display is connected to planning software and illuminates the correct hole. In an empirical evaluation with physicians and researchers from the medical domain, we compare the illuminated against the conventional template as used in magnetic resonance imaging (MRI) guided prostate biopsies. Our results show that the EL display significantly improves task completion time by 51%, task load by 47% and usability by 30%.

In this paper, we present results from an ongoing participatory design research project that focuses on the design and evaluation of a tagging-system and corresponding visualizations to support actors and visitors of an urban gardening project. The contributions of this work are threefold. First, it addresses the yet underexplored field of integrating context information beyond location and time into mobile tagging systems and folksonomies. Second, it suggests novel visualizations to explore tagged data and folksonomies beyond tag clouds and simple map representations on mobile phones. And third, it gives novel insights into supporting the embedded practices of a DIY (Do-It-Yourself) urban gardening community with interactive systems based on ethnographic fieldwork and participatory design.

In this paper, we present the novel concept of ephemeral user interfaces. Ephemeral user interfaces contain at least one user interface (UI) element that is intentionally created to last for a limited time only and typically incorporate materials that evoke a rich and multisensory perception, such as water, fire, soap bubbles or plants. We characterize the term "ephemeral user interface" and, based on a review of existing user interfaces that fall into this research area but have not been discussed under one common term before, we present a design space for ephemeral user interfaces providing a terminology for (a) materials for ephemeral UI elements, (b) interaction and (c) aspects of ephemerality. This paper contributes to the ongoing research on materiality of user interfaces as well as on conceptualizing visionary interaction styles with novel materials.

Natural user interfaces by means of gesture and speech interaction have become a hot topic in research as well as already for real products. Most use cases currently center around consumer electronics devices like smart phones, TV sets, gaming, or other large screens like tabletops.

Cars offer an increasing number of infotainment systems as well as comfort functions that can be controlled by the driver. In our research, we investigate new interaction techniques that aim to make it easier to interact with these systems while driving. We suggest utilizing the steering wheel as an additional interaction surface. In this paper, we present two user studies conducted with a working prototype of a multi-touch steering wheel. In the first, we developed a user-defined steering wheel gesture set, and in the second, we applied the identified gestures and compared their application to conventional user interaction with infotainment systems in terms of driver distraction. The main outcome was that driver's visual demand is reduced significantly by using gestural interaction on the multi-touch steering wheel.

This paper presents an overview of an ongoing doctoral research project on materiality and tangible user interfaces (TUIs). The aim of the thesis is to address materiality as an important design element of tangible user interfaces and develop a material-based design knowledge for TUIs, e.g., in the form of material-based design patterns and evaluation methods. This is done iteratively in three steps: (1) compilation of interdisciplinary material theories and their application to tangible user interfaces, (2) exploratory prototyping of material-based interaction, and (3) validation of developed design knowledge and evaluation techniques in evaluations of TUI prototypes in different application domains.

In this paper, we explore the potential of combining shared and interactive displays (e.g. a multi-touch table) with personal devices (e.g. mobile phones) as an important class of heterogeneous multi-display environments. Within six case studies applications and interactions were invented and implemented that utilize the potential of such heterogeneous multi-display environments. We were in particular interested how to design systems that include interaction across different displays and how to manage public and private information in a group setting. One case study, a digital card game, highlights these design challenges. We explore different natural ways of interaction, including touching the table as well as motion gestures with mobile phones. With this application we provide a use case to discuss gestures combining mobile phones with tabletop surfaces, as well as to explore a private-public display setting. First results showed that combining tables and mobile phones provide a suitable and understandable way for interaction in these settings.

In this paper we inquire material qualities for tangible user interfaces and introduce the term ephemeral user interface. We present an example user interface that employs fragile soap bubbles for human-computer interaction: the user has to move or blow bubbles over a dark liquid surface in order to interact with a computer system. Our installation uses liquid, smoke, and soap bubbles as unusual and transient materials that demand a very cautious and calm handling, and furthermore elude the complete control of users, as the bubbles inevitably burst after a short while. Thus, the soap bubble interface provokes thoughts about durability, control, and materiality of tangible user interfaces.

In this paper, we introduce the design and implementation of a digital card game that combines mobile phone gestures and multi-touch tabletop interactions. Conceptually based on interactions in traditional card games, our Poker game application explores different natural ways of interaction, including touching the table as well as tilting, throwing, and shaking a mobile phone. By exemplarily translating traditional gestures into the digital domain, we provide a use case to discuss useful gestures combing mobile phones with tabletop surfaces, as well as to explore a private-public display setting for digital card games at interactive tables. Additionally, the mobile phone provides a tangible feeling similar to physical cards. The preliminary user study showed that users preferred using mobile phones for interaction compared to direct interaction on the multi-touch table.

This paper presents art historical research and education as a novel application area for tangible user interfaces. The academic discipline of art history and its subjects are currently undergoing changes that will lead to a rising importance of computers. However, the computer is generally not the art historian's tool of choice. We feel that this is due to existing GUI systems not fully meeting researchers' needs. We therefore propose a design for a tabletop tangible user interface considering art historians' desire "to collect things as tokens" [1] and to remain within traditional techniques. We present a case study of the usage of image cards within iconographic work. Based on our results, we derive implications for the design of the tangible interface that integrates approved traditional paper based techniques with the advantages of digital representation.