Conventional navigation systems, fixated on metrics such as time and distance, neglect the driver’s emotional well-being, despite driving routes being inherent emotional triggers. This raises a critical question for the Intelligent User Interface community: How can intelligent systems successfully route information based on emotion? To address this gap, we introduce HappyRouting, an empathic car interface designed as an initial attempt to guide drivers through real-world traffic while actively optimizing for positive emotional states. Our core technical contribution is a machine learning-based emotion map layer that predicts the affective valence along various routes using both static and dynamic contextual data. HappyRouting enables the generation of “happy routes” integrated into a functional vehicular interface prototype. We explored the efficacy of this approach in a preliminary, small-scale driving study (N = 13). Our initial findings provide provocative evidence: Emotion-optimized routes successfully increased the subjectively perceived valence by 11% (p =.007) compared to standard routes. Furthermore, despite taking 1.25 times longer on average, participants consistently perceived the travel duration as shorter. This result suggests that integrating emotional optimization could fundamentally challenge the speed-first paradigm. However, recognizing the constraints of our initial, limited sample, we conclude by discussing ethical and computational challenges that must be resolved before emotion-based routing can be safely and scalably integrated into next-generation intelligent navigation apps.

Inspired by the concepts of diminishing reality and ad-blocking in browsers, this study investigates the perceived benefits and concerns of blocking physical, real-world content, particularly ads, through Extended Reality (XR). To understand how users perceive this concept, we first conducted a user study (n=18) with an adblocking prototype to gather initial insights. The results revealed a mixed willingness to adopt XR blockers, with participants appreciating aspects such as customizability, convenience, and privacy. Expected benefits included enhanced focus and reduced stress, while concerns centered on missing important information and increased feelings of isolation. Hence, we investigated the user acceptance of different ad-blocking visualizations through a follow-up online survey (n=120), comparing six concepts based on related work. The results indicated that the XR ad-blocker visualizations play a significant role in how and for what kinds of advertisements such a concept might be used, paving the path for future feedback-driven prototyping.

With the recent advancements in Large Language Models (LLMs), web developers increasingly apply their code-generation capabilities to website design. However, since these models are trained on existing designerly knowledge, they may inadvertently replicate bad or even illegal practices, especially deceptive designs (DD). This paper examines whether users can accidentally create DD for a fictitious webshop using GPT-4. We recruited 20 participants, asking them to use ChatGPT to generate functionalities (product overview or checkout) and then modify these using neutral prompts to meet a business goal (e.g., “increase the likelihood of us selling our product”). We found that all 20 generated websites contained at least one DD pattern (mean: 5, max: 9), with GPT-4 providing no warnings. When reflecting on the designs, only 4 participants expressed concerns, while most considered the outcomes satisfactory and not morally problematic, despite the potential ethical and legal implications for end-users and those adopting ChatGPT’s recommendations.

Publisher Copyright: © 2025 Copyright held by the owner/author(s).

Figure 1: Our study explores user perceptions of web-browser tracking protection plugins under the three conditions no plugin, functional plugin, and placebo plugin, where users were only primed with narratives without altering the website.During a hotel booking task, users felt more protected with functional or placebo plugins despite no actual changes to the website, revealing the participant's inability to judge tracking protection effectiveness accurately.We generated the figure using DALL-E.

Personas offer an empathetic approach to capturing user requirements, translating user needs into relatable narratives. However, creating personas manually is time-consuming. Large Language Models (LLMs) can generate personas with convincing natural language, challenging traditional methods. Yet, LLM-generated personas may reflect biases from their training data, potentially compromising diversity. Our study explores how diversity is considered in LLM-generated personas through a qualitative user study with 22 participants. Participants generated personas without specific diversity prompts in the first task, revealing how users naturally interact with LLMs. In the second task, participants were explicitly asked to consider diversity aspects when prompting for personas. Analyzing the prompts and outputs showed that users tend to request less diversity unless explicitly instructed. Meanwhile, LLMs can introduce diversity even when not prompted, potentially broadening representation. However, we also found a critical pitfall: LLM-generated personas may appear diverse due to mentioning various aspects, but fail to translate them into meaningful implications for requirements engineering. This shows the need for a more deliberate approach when using LLMs for persona creation to ensure diversity is not just performative but genuinely informative for design and development.

Cycling has gained popularity due to growing interest in healthy and sustainable lifestyles. Simultaneously, Augmented Reality (AR) Head-Mounted Displays (HMDs) can assist cyclists by presenting notifications within their field of view without diverting their attention to external devices. While previous studies have investigated these advantages, safety concerns have primarily limited them to lab settings, creating a notable gap in understanding their real-world feasibility. We conducted a user study with 20 participants on a shared-use outdoor path and explored the impact of text-based HMD notification placement (top, right, bottom), on attentional tunneling and cyclists' experiences. Our results suggested that while the bottom placement received higher scores for perceived safety and noticeability, HMD notifications induced attentional tunneling, regardless of placement. We discuss our findings and present design insights for future HMD systems aimed at enhancing cyclists' safety and experience.

The rise of Large Language Models (LLMs) in education introduced powerful tools such as ChatGPT, which students increasingly use for academic purposes. However, these technologies present significant challenges for higher education, such as the risk of undermining academic integrity through AI-assisted performance and the uncertainty around proper use. This paper seeks to understand the benefits and risks university students perceive regarding LLM usage and how those influence their acceptance and use of related services in higher education. To this end, we employed a mixed-method approach. Using the UTAUT2 model extended by a Risk Expectancy construct, we conducted an online survey and follow-up interviews with university students. The results indicated that while students perceive considerable risks related to LLMs, those do not impact their usage and behavioral intention. We discuss this phenomenon based on the qualitative interview analysis and suggest research directions for future work.

User interaction with digital systems requires Fine Motor Control (FMC), especially if the interfaces are complex or require high fidelity and fine-grained interactions. Despite its importance, Fine Motor Control is often overlooked in interactive system design, partly because of its complex assessment. Measuring changes in fine motor abilities due to prolonged use or fatigue currently requires repeated manual testing. This paper analyzes the concept of using the digital mobile devices’ input behavior to assess the user’s Fine Motor Control. For this, we show that Fine Motor Control can be assessed for touch and stylus-based interaction with a digital mobile system. We conducted a user study, where participants performed a Nine Hole Peg Test and a predefined Copy Drawing Test before and after exercises that affect fine motor skills. Based on this data, we investigated how metrics such as pressure, velocity, and entropy for touch and stylus input can be used to predict Fine Motor Control.

Current research in Mixed Reality (MR) presents a wide range of novel use cases for blending virtual elements with the real world. This yet-to-be-ubiquitous technology challenges how users currently work and interact with digital content. While offering many potential advantages, MR technologies introduce new security, safety, and privacy challenges. Thus, it is relevant to understand users’ apprehensions towards MR technologies, ranging from security concerns to social acceptance. To address this challenge, we present the Mixed Reality Concerns (MRC) Questionnaire, designed to assess users’ concerns towards MR artifacts and applications systematically. The development followed a structured process considering previous work, expert interviews, iterative refinements, and confirmatory tests to analytically validate the questionnaire. The MRC Questionnaire offers a new method of assessing users’ critical opinions to compare and assess novel MR artifacts and applications regarding security, privacy, social implications, and trust.

Heightened AI expectations facilitate performance in human-AI interactions through placebo effects. While lowering expectations to control for placebo effects is advisable, overly negative expectations could induce nocebo effects. In a letter discrimination task, we informed participants that an AI would either increase or decrease their performance by adapting the interface, when in reality, no AI was present in any condition. A Bayesian analysis showed that participants had high expectations and performed descriptively better irrespective of the AI description when a sham-AI was present. Using cognitive modeling, we could trace this advantage back to participants gathering more information. A replication study verified that negative AI descriptions do not alter expectations, suggesting that performance expectations with AI are biased and robust to negative verbal descriptions. We discuss the impact of user expectations on AI interactions and evaluation.

Electromyography (EMG) enables hands-free interactions by detecting muscle activity at different human body locations. Previous studies have demonstrated that input performance based on isometric contractions is muscle-dependent and can benefit from synchronous biofeedback. However, it remains unknown whether stimulation before interaction can help to localize and tense a muscle faster. In a response-based VR experiment (N=21), we investigated whether prior stimulation using visual or tactile cues at four different target muscles (biceps, triceps, upper leg, calf) can help reduce the time to perform isometric muscle contractions. The results show that prior stimulation decreases EMG reaction times with visual, vibrotactile, and electrotactile cues. Our experiment also revealed important findings regarding learning and fatigue at the different body locations. We provide qualitative insights into the participants’ perceptions and discuss potential reasons for the improved interaction. We contribute with implications and use cases for prior stimulated muscle activation.

When using Head-Mounted Displays (HMDs), users may not always notice or report visual discomfort by blurred vision through unadjusted lenses, motion sickness, and increased eye strain. Current measures for visual discomfort rely on users’ self-reports those susceptible to subjective differences and lack of real-time insights. In this work, we investigate if Electroencephalography (EEG) can objectively measure visual discomfort by sensing Event-Related Potentials (ERPs). In a user study ($\mathrm{N}=20$), we compare four different levels of Gaussian blur in a user study while measuring ERPs at occipito-parietal EEG electrodes. The findings reveal that specific ERP components (i.e., P1, N2, and P3) discriminated discomfort-related visual stimuli and indexed increased load on visual processing and fatigue. We conclude that time-locked brain activity can be used to evaluate visual discomfort and propose EEG-based automatic discomfort detection and prevention tools.

Figure 1: Mixed Reality devices could potentially warn its users about fall hazards such as staircases.In our explorative prototype study, participants tested three different warning types: A superimposed small line at the stair landing (Left), a bigger warning including a text label (Middle), and a full-scale warning blocking the whole stair entrance including a text label (Right).

The quality and readability of an individual's handwriting and drawing can be influenced by various factors, including their level of physical exertion.This enables us to explore the quantification of exertion by observing an individual's handwriting.To test this hypothesis, we collected data from 17 participants, building a database of handwriting and drawing samples and their corresponding Borg 10 exertion ratings at the time of drawing.In this paper, we investigate using machine learning techniques to estimate perceived exertion before, during, and after physical activity based on handwriting and drawings.We apply a regression model to compare different drawing tasks and demonstrate that perceived exertion can be predicted using simple line drawings.However, more complex sketches and handwriting demand further research.Our findings suggest that interactive systems could use handwriting and drawing to intervene when users experience excessive discomfort.

Rapid Serial Visual Presentation (RSVP) improves the reading speed for optimizing the user's information processing capabilities on Virtual Reality (VR) devices. Yet, the user's RSVP reading performance changes over time while the reading speed remains static. In this paper, we evaluate pupil dilation as a physiological metric to assess the mental workload of readers in real-time. We assess mental workload under different background lighting and RSVP presentation speeds to estimate the optimal color that discriminates the pupil diameter varying RSVP presentation speeds. We discovered that a gray background provides the best contrast for reading at various presentation speeds. Then, we conducted a second study to evaluate the classification accuracy of mental workload for different presentation speeds. We find that pupil dilation relates to mental workload when reading with RSVP. We discuss how pupil dilation can be used to adapt the RSVP speed in future VR applications to optimize information intake.

Haptic feedback reportedly enhances human interaction with 3D data, particularly improving the retention of mental representations of digital objects in immersive settings. However, the effectiveness of visuohaptic integration in promoting object retention across different display environments remains underexplored. Our study extends previous research on the retention effects of haptics from virtual reality to a projected surface display to assess whether earlier findings generalize to 2D environments. Participants performed a delayed match-to-sample task incorporating visual, haptic, and visuohaptic sensory feedback within a projected surface display environment. We compared error rates and response times across these sensory modalities and display environments. Our results reveal that visuohaptic integration significantly enhances object retention on projected surfaces, benefiting task performance across display environments. Our findings suggest that haptics can improve object retention without requiring fully immersive setups, offering insights for the design of interactive systems that assist professionals who rely on precise mental representations of digital objects.

From carrying grocery bags to holding onto handles on the bus, there are a variety of situations where one or both hands are busy, hindering the vision of ubiquitous interaction with technology. Voice commands, as a popular hands-free alternative, struggle with ambient noise and privacy issues. As an alternative approach, research explored movements of various body parts (e.g., head, arms) as input modalities, with foot-based techniques proving particularly suitable for hands-free interaction. Whereas previous research only considered the movement of the foot as a whole, in this work, we argue that our toes offer further degrees of freedom that can be leveraged for interaction. To explore the viability of toe-based interaction, we contribute the results of a controlled experiment with 18 participants assessing the impact of five factors on the accuracy, efficiency and user experience of such interfaces. Based on the findings, we provide design recommendations for future toe-based interfaces.

Mid-air gestures, widely used in today’s Augmented Reality (AR) applications, are prone to the “gorilla arm” effect, leading to discomfort with prolonged interactions. While prior work has proposed metrics to quantify this effect and means to improve comfort and ergonomics, these works usually only consider simplistic, one-dimensional AR interactions, like reaching for a point or pushing a button. However, interacting with AR environments also involves far more complex tasks, such as rotational knobs, potentially impacting ergonomics. This paper advances the understanding of the ergonomics of rotational mid-air interactions in AR. For this, we contribute the results of a controlled experiment exposing the participants to a rotational task in the interaction space defined by their arms’ reach. Based on the results, we discuss how novel future mid-air gesture modalities benefit from our findings concerning ergonomic-aware rotational interaction.

This paper analyses Human-Computer Interaction (HCI) literature reviews to provide a clear conceptual basis for authors, reviewers, and readers. HCI is multidisciplinary and various types of literature reviews exist, from systematic to critical reviews in the style of essays. Yet, there is insufficient consensus of what to expect of literature reviews in HCI. Thus, a shared understanding of literature reviews and clear terminology is needed to plan, evaluate, and use literature reviews, and to further improve review methodology. We analysed 189 literature reviews published at all SIGCHI conferences and ACM Transactions on Computer-Human Interaction (TOCHI) up until August 2022. We report on the main dimensions of variation: (i) contribution types and topics; and (ii) structure and methodologies applied. We identify gaps and trends to inform future meta work in HCI and provide a starting point on how to move towards a more comprehensive terminology system of literature reviews in HCI.

Eye tracking is increasingly becoming prevalent for health-related interactive systems. Eye tracking can automatically reveal the presence of Central Field Loss (CFL), a dysfunctional visual behavior requiring time-intensive medical assessments. Since CFL typically results in poor fixation stability and more frequent saccades, this work investigates the use of machine learning to estimate the likelihood of CFL based on eye-movement data. We compared random forests, support vector machines, and long-short-term memory (LSTM) neural networks for their ability to discriminate between the presence or absence of an experimentally-induced CFL. We found that the estimation accuracy increases with larger samples of eye-tracking data. However, the computational costs outweigh any increase in accuracy after classifying window sizes of 1600 msec. Here, traditional machine learning approaches outperform the LSTM neural network. We discuss implications for continuous end-user CFL monitoring and processing power to provide an outlook for gaze-based wearable health devices in human-computer interaction.

Pilots in non-commercial aviation have minimal access to digital support tools. Common 2D maps, displayed on tablets, are often the only digital information source that fails to adequately capture the 3D airspace and its surroundings, challenging the pilot’s workload and awareness. In this work, we developed and tested a Mixed Reality (MR) prototype with twelve General Aviation (GA) pilots using a full-sized flight simulator environment. The prototype’s demonstration showcased the capabilities of contemporary technology and its potential applications. Following the simulation, in-depth interviews were conducted with the participating pilots to discern their perspectives on integrating MR solutions into cockpit environments. The study revealed valuable insights into pilots’ concerns, design prerequisites for future systems, and potential use cases. This work not only highlights the feasibility of MR implementations but also provides a foundation for the development of enhanced digital tools for GA, aiming to alleviate pilot workload and augment situational awareness.

Exergames, video games designed to blend entertainment with physical activity, aim to improve users’ physical fitness by combining gaming with exercise. However, integrating exercise equipment, such as rowers, bikes, and ski ergometers into Virtual Reality (VR) environments remains challenging. In this poster, we introduce a toolkit that simplifies the integration of ergometers into Unity-based projects. Researchers can access detailed ergometer data for logging and use inside their projects, while our toolkit handles tedious tasks, like connection-handling or parsing. VRow offers valuable support for creating immersive and interactive fitness experiences.

The parallel improvements in multi-material 3D printers and the quality of conductive filament open new possibilities for the fabrication of tangible and functional objects. In this studio, we discuss best practices for 3D printed electronics, talk about encountered problems, and derive design recommendations. We will guide the participants through a fabrication process by practically designing and printing objects. Consequently, we contemplate individual functional fabricated components, including small printed circuits and multi-material prints. We aim to spark a discussion about individually experienced challenges participants encountered during their design and fabrication process. This discussion includes problem-solving strategies, whose insights benefit other participants. Finally, we show the potential of printed electronics and discuss encouraging new opportunities in this field.

Virtual reality experiences increasingly use physiological data for virtual environment adaptations to evaluate user experience and immersion. Previous research required complex medical-grade equipment to collect physiological data, limiting real-world applicability. To overcome this, we present SensCon for skin conductance and heart rate data acquisition. To identify the optimal sensor location in the controller, we conducted a first study investigating users' controller grasp behavior. In a second study, we evaluated the performance of SensCon against medical-grade devices in six scenarios regarding user experience and signal quality. Users subjectively preferred SensCon in terms of usability and user experience. Moreover, the signal quality evaluation showed satisfactory accuracy across static, dynamic, and cognitive scenarios. Therefore, SensCon reduces the complexity of capturing and adapting the environment via real-time physiological data. By open-sourcing SensCon, we enable researchers and practitioners to adapt their virtual reality environment effortlessly. Finally, we discuss possible use cases for virtual reality-embedded physiological sensing.

Racket sports offer an enjoyable form of physical activity and are fertile ground for interactive technologies supporting new players. Yet, current research has neglected its potential to support not only active players but also coaches in their training methods. To investigate how interactive technologies can support skill acquisition in training, we designed an Electrical Muscle Stimulation (EMS) system that helps maintain the ready position in crossminton. We compared the system with a vibrotactile solution in a user study, interviewing novice players and experienced coaches about their perception of the system. The system allowed coaches to effectively and immediately guide players to the ready position. An EMS-based feedback system for coaches can potentially reduce delay (physical and cognitive) for trainees, as stated by coaches. Our work contributes insights into designing systems that facilitate learning sports techniques using interactive feedback.

Exploring and interacting with electronics is challenging as the internal processes of components are not visible. Further barriers to engagement with electronics include fear of injury and hardware damage. In response, Augmented Reality (AR) applications address those challenges to make internal processes and the functionality of circuits visible. However, current apps are either limited to abstract low-fidelity applications or entirely virtual environments. We present ElectronicsAR, a tangible high-fidelity AR electronics kit with scaled hardware components representing the shape of real electronics. Our evaluation with 24 participants showed that users were more efficient and more effective at naming components, as well as building and debugging circuits. We discuss our findings in the context of ElectronicsAR's unique characteristics that we contrast with related work. Based on this, we discuss opportunities for future research to design functional mobile AR applications that meet the needs of beginners and experts.

Novel input methods for game design often excite users, especially if they extend the way one interacts with the system. Electromyography (EMG) has the inherent potential to provide an intuitive - yet challenging - input channel for interactive systems. While this difficulty in control often limits the scope of applications for EMG in most systems, we argue that these qualities are especially relevant for games and playful interaction. The inherently challenging qualities of EMG input make the modality a prime candidate for designing body-centric playful experiences. Yet, we still need to understand its limitations to create engaging rather than frustrating experiences for users. In this work, we investigate EMG's potential to support playful interaction through exploratory studies, deriving feasible game interactions based on EMG's technical constraints, and study their application in game design. Based on our findings, we highlight design implications and pitfalls to avoid when creating EMG-based entertainment systems.

Cyclists' attention is often compromised when interacting with notifications in traffic, hence increasing the likelihood of road accidents. To address this issue, we evaluate three notification interaction modalities and investigate their impact on the interaction performance while cycling: gaze-based Dwell Time, Gestures, and Manual And Gaze Input Cascaded (MAGIC) Pointing. In a user study (N=18), participants confirmed notifications in Augmented Reality (AR) using the three interaction modalities in a simulated biking scenario. We assessed the efficiency regarding reaction times, error rates, and perceived task load. Our results show significantly faster response times for MAGIC Pointing compared to Dwell Time and Gestures, while Dwell Time led to a significantly lower error rate compared to Gestures. Participants favored the MAGIC Pointing approach, supporting cyclists in AR selection tasks. Our research sets the boundaries for more comfortable and easier interaction with notifications and discusses implications for target selections in AR while cycling.

In medicine, patients can obtain real benefits from a sham treatment. These benefits are known as the placebo effect. We report two experiments (Experiment I: N = 369; Experiment II: N = 100) demonstrating a placebo effect in adaptive interfaces. Participants were asked to solve word puzzles while being supported by no system or an adaptive AI interface. All participants experienced the same word puzzle difficulty and had no support from an AI throughout the experiments. Our results showed that the belief of receiving adaptive AI support increases expectations regarding the participant’s own task performance, sustained after interaction. These expectations were positively correlated to performance, as indicated by the number of solved word puzzles. We integrate our findings into technological acceptance theories and discuss implications for the future assessment of AI-based user interfaces and novel technologies. We argue that system descriptions can elicit placebo effects through user expectations biasing the results of user-centered studies.

Learning a musical instrument requires regular exercise. However, students are often on their own during their practice sessions due to the limited time with their teachers, which increases the likelihood of mislearning playing techniques. To address this issue, we present Let’s Frets - a modular guitar learning system that provides visual indicators and capturing of finger positions on a 3D-printed capacitive guitar fretboard. We based the design of Let’s Frets on requirements collected through in-depth interviews with professional guitarists and teachers. In a user study (N=24), we evaluated the feedback modules of Let’s Frets against fretboard charts. Our results show that visual indicators require the least time to realize new finger positions while a combination of visual indicators and position capturing yielded the highest playing accuracy. We conclude how Let’s Frets enables independent practice sessions that can be translated to other musical instruments.

Physical exercises can benefit our health, but avoiding improper form and overexertion is essential. Facilitating bodily insights can encourage learning about exercise form, allowing users to gain a deeper understanding of their physiology. To investigate this, we conducted a lab experiment where amateur users performed bicep curls, and interviews with sports coaches. Participants were provided with FitBack—a system that monitors muscle activity during exercises via electromyography (EMG) and offers real-time biofeedback. Amateurs reported that they were successful in improving their exercise form and could acquire deeper bodily insights. Coaches reflected on how understanding muscle activity through EMG could be effectively used for increasing body awareness during coaching, highlighting that EMG-based biofeedback is beneficial for a diverse set of users. Our work contributes insights into using bodily sensing to help users understand their bodies. We contribute guidelines for designing systems that use EMG biofeedback effectively in physical activity.

Due to recent technological advancements, governments can exercise an unprecedented amount of power over their citizens. It is vital to understand how governments impose restrictions on citizens through digital technologies, especially if those restrictions can save lives. Here, we look at the case of Home Quarantine—a Polish government-mandated smartphone application whose use has become mandatory during the COVID-19 crisis. Users in quarantine are legally required to use the application, reporting location and take selfie photographs. We conducted an autoethnographic diary study of one author using the app during quarantine and interviews with 23 users. We found that the app assisted in creating quarantine life routines and affected social interactions. Users connected with the world outside quarantine through selfies. We also uncovered key reservations our users had about using this app. Our work broadens the understanding of location-based apps and practices around surveillance technologies.

While fitness trackers are increasingly popular among users, recent studies have shown that the health benefits of wearing a tracker are not apparent. The need to explicitly retrieve data can lead to limited benefits. Understanding how users can access, understand, and reflect on their data can lead to building systems that benefit our wellbeing. In this work, we explore the feasibility of using ubiquitous artifacts for unobtrusive feedback in health tracking. We evaluated a concept based on design dimensions for personal visualization on a smart mirror in a user study. Our design puts emphasis on the temporality of presented data. Participants found the visualizations comprehensive, rating cardiac and inertial data most useful as well as approved the different levels of temporal aggregation. Our work contributes findings on how to represent health-related data with ubiquitous artifacts to increase users’ awareness.

A steadily increasing number of notifications, auditory, visual or ambient, competes for the user’s attention. Frequent unsuitable notifications can lead to a breakdown in efficiency and increase error rates. This paper compares the effectiveness, disruptiveness, and user experience of three different notification modalities: On-Object, ambient On-Environment, and On-Smartphone notifications. In a user study with 24 participants, we evaluate the three notification modalities during a cooking task where users were frequently exposed to notifications. Our results show that ambient On-Environment notifications minimize the time in which users can resume their primary task. Ambient On-Environment notifications were also perceived as least disrupting compared to the other two notification modalities. We discuss the design requirements for non-disruptive notifications in smart home environments and conclude with future strategies for notifying users at different urgency levels.

Detecting emotions while driving remains a challenge in Human-Computer Interaction. Current methods to estimate the driver’s experienced emotions use physiological sensing (e.g., skin-conductance, electroencephalography), speech, or facial expressions. However, drivers need to use wearable devices, perform explicit voice interaction, or require robust facial expressiveness. We present VEmotion (Virtual Emotion Sensor), a novel method to predict driver emotions in an unobtrusive way using contextual smartphone data. VEmotion analyzes information including traffic dynamics, environmental factors, in-vehicle context, and road characteristics to implicitly classify driver emotions. We demonstrate the applicability in a real-world driving study (N = 12) to evaluate the emotion prediction performance. Our results show that VEmotion outperforms facial expressions by 29% in a person-dependent classification and by 8.5% in a person-independent classification. We discuss how VEmotion enables empathic car interfaces to sense the driver’s emotions and will provide in-situ interface adaptations on-the-go.

Virtual Reality (VR) experiences have massively improved in the mediation of feedback. However, the simulation of forces is still limited. This paper presents Odin's Helmet, a head-worn device to simulate g-forces that act on the human head in real-life situations. Odin's Helmet uses four head-mounted propellers as actuators to simulate g-forces through pushing and pulling the user's head while being immersed in VR. Odin's Helmet's goal is to increase presence and manipulate the user's perception of the otolith organ in the vestibular system. The user's perception will be tricked to experience a sensation of self-movement in VR. A technical evaluation shows Odin's Helmet's applicability to apply perceivable g-forces to the user's head. We conclude with future use cases of Odin's Helmet, such as redirected walking by controlling the user's head orientation, attention guidance, and wind simulations through Odin's Helmet.

Improvising on the piano keyboard requires extensive skill development, which may reduce the feeling of immersion and flow for amateur players. However, being able to add simple musical effects greatly boosts a player's ability to express their unique playing style. To simplify this process, we designed an electromyography-based (EMG) system which integrates seamlessly into normal play by allowing musicians to modulate sound pitch using their thumb. We conducted an exploratory user study where users played a predefined melody and improvised using our system and a standard pitch wheel. Interview responses and survey answers showed that the EMG-based system supported the players' musical flow. Additionally, interviews indicated the system's capabilities to foster player creativity, and that players enjoyed experimenting with the effect. Our work illustrates how EMG can support seamless integration into existing systems to extend the range of interactions provided by a given interface.

Exploiting emotions for user interface evaluation became an increasingly important research objective in Human-Computer Interaction. Emotions are usually assessed through surveys that do not allow information to be collected in real-time. In our work, we suggest the use of smartphones for mobile emotion assessment. We use the front-facing smartphone camera as a tool for emotion detection based on facial expressions. Such information can be used to reflect on emotional states or provide emotion-aware user interface adaptation. We collected facial expressions along with app usage data in a two-week field study consisting of a one-week training phase and a one-week testing phase. We built and evaluated a person-dependent classifier, yielding an average classification improvement of 33% compared to classifying facial expressions only. Furthermore, we correlate the estimated emotions with concurrent app usage to draw insights into changes in mood. Our work is complemented by a discussion of the feasibility of probing emotions on-the-go and potential use cases for future emotion-aware applications.

Rapid Serial Visual Presentation (RSVP) has gained popularity as a method for presenting text on wearable devices with limited screen space. Nonetheless, it remains unclear how to calibrate RSVP display parameters, such as spatial alignments or presentation rates, to suit the reader's information processing ability at high presentation speeds. Existing methods rely on comprehension and subjective workload scores, which are influenced by the user's knowledge base and subjective perception. Here, we use electroencephalography (EEG) to directly determine how individual information processing varies with changes in RSVP display parameters. Eighteen participants read text excerpts with RSVP in a repeated-measures design that manipulated the Text Alignment and Presentation Speed of text representation. We evaluated how predictive EEG metrics were of gains in reading speed, subjective workload, and text comprehension. We found significant correlations between EEG and increasing Presentation Speeds and propose how EEG can be used for dynamic selection of RSVP parameters.

The pervasive availability of media in foreign languages is a rich resource for language learning. However, learners are forced to interrupt media consumption whenever comprehension problems occur. We present BrainCoDe, a method to implicitly detect vocabulary gaps through the evaluation of event-related potentials (ERPs). In a user study (N=16), we evaluate BrainCoDe by investigating differences in ERP amplitudes during listening and reading of known words compared to unknown words. We found significant deviations in N400 amplitudes during reading and in N100 amplitudes during listening when encountering unknown words. To evaluate the feasibility of ERPs for real-time applications, we trained a classifier that detects vocabulary gaps with an accuracy of 87.13% for reading and 82.64% for listening, identifying eight out of ten words correctly as known or unknown. We show the potential of BrainCoDe to support media learning through instant translations or by generating personalized learning content.

Human vision underlies several constraints in its spatial, temporal and spectral resolution. One limitation is the visual sensation of temperatures, which can be a useful addition for human sight in education or work scenarios. This demo presents how mixed reality applications can virtually superimpose objects’ thermal properties with the physical world into a coherent environment. We present a prototype that utilizes either a smartphone or a head-mounted display to mediate thermal flux. We show the feasibility of our prototype in the context of a physics lab experiment that fosters the understanding of heat conduction for different materials. We envision thermal vision as an addition to the human sight that can be accomplished with already available technologies. Finally, we discuss how our prototype can be designed as a context-aware in-situ interface that toggles between regular and thermal vision.

We present DronOS, a rapid prototyping framework that can track, control, and automate drone routines. Previous research in the domain of Human-Drone Interaction relied on hardware or proprietary vendor-dependent libraries that had to be exclusively programmed for specific use cases. This forces users to stick with a drone manufacturer or model as well as limiting users in transferring their drone control logic to other drones. To overcome the aforementioned issues, our framework uses low-cost off-the-shelf hardware and applies to a variety of already available or self-crafted drones. To assess the usability of DronOS, we evaluate three drone programming modes: Unity Scripting, Vive Scripting, and Vive Realtime. We find that Vive Scripting required the least subjective workload in programming drone routines while Unity Scripting yielded the highest accuracy and Vive Realtime the least task completion time. We anticipate requirements for drone prototyping frameworks that target novice and expert users as operators.

Mastering fine motor tasks, such as playing the guitar, takes years of time-consuming practice. Commonly, expensive guidance by experts is essential for adjusting the training program to the student's proficiency. In our work, we showcase the suitability of Electromyography to detect fine-grained hand and finger postures in an exemplary guitar tutor scenario. We present EMGuitar, an interactive guitar tutoring system, that assists students by reporting on play correctness and adjusts playback tempi automatically. We report person-dependent classification utilizing a ring of electrodes around the forearm with an F1 score of up to 0.89 on recorded calibration data. Furthermore, our system was received well by neither diminishing ease of use nor being disruptive for the participants. Based on the received comments, we identified the need for detailed play accuracy feedback down to individual chords, for which we suggest an adapted visualization and an algorithmic approach.

A common objective for context-aware computing systems is to predict how user interfaces impact user performance regarding their cognitive capabilities. Existing approaches such as questionnaires or pupil dilation measurements either only allow for subjective assessments or are susceptible to environmental influences and user physiology. We address these challenges by exploiting the fact that cognitive workload influences smooth pursuit eye movements. We compared three trajectories and two speeds under different levels of cognitive workload within a user study (N=20). We found higher deviations of gaze points during smooth pursuit eye movements for specific trajectory types at higher cognitive workload levels. Using an SVM classifier, we predict cognitive workload through smooth pursuit with an accuracy of 99.5% for distinguishing between low and high workload as well as an accuracy of 88.1% for estimating workload between three levels of difficulty. We discuss implications and present use cases of how cognition-aware systems benefit from inferring cognitive workload in real-time by smooth pursuit eye movements.

Individuals with cognitive impairments currently leverage extensive human resources during their transitions from assisted living to independent living. In Western Europe, many government-supported volunteer organizations provide sheltered living facilities; supervised environments in which people with cognitive impairments collaboratively learn daily living skills. In this paper, we describe communal cooking practices in sheltered living facilities and identify opportunities for supporting these with interactive technology to reduce volunteer workload. We conducted two contextual observations of twelve people with cognitive impairments cooking in sheltered living facilities and supplemented this data through interviews with four employees and volunteers who supervise them. Through thematic analysis, we identified four themes to inform design requirements for communal cooking activities: Work organization, community, supervision, and practicalities. Based on these, we present five design implications for assistive systems in kitchens for people with cognitive deficiencies.

Touchscreens are the most successful input method for smartphones. Despite their flexibility, touch input is limited to the location of taps and gestures. We present PalmTouch, an additional input modality that differentiates between touches of fingers and the palm. Touching the display with the palm can be a natural gesture since moving the thumb towards the device's top edge implicitly places the palm on the touchscreen. We present different use cases for PalmTouch, including the use as a shortcut and for improving reachability. To evaluate these use cases, we have developed a model that differentiates between finger and palm touch with an accuracy of 99.53% in realistic scenarios. Results of the evaluation show that participants perceive the input modality as intuitive and natural to perform. Moreover, they appreciate PalmTouch as an easy and fast solution to address the reachability issue during one-handed smartphone interaction compared to thumb stretching or grip changes.

We present VRHapticDrones, a system utilizing quadcopters as levitating haptic feedback proxy. A touchable surface is attached to the side of the quadcopters to provide unintrusive, flexible, and programmable haptic feedback in virtual reality. Since the users' sense of presence in virtual reality is a crucial factor for the overall user experience, our system simulates haptic feedback of virtual objects. Quadcopters are dynamically positioned to provide haptic feedback relative to the physical interaction space of the user. In a first user study, we demonstrate that haptic feedback provided by VRHapticDrones significantly increases users' sense of presence compared to vibrotactile controllers and interactions without additional haptic feedback. In a second user study, we explored the quality of induced feedback regarding the expected feeling of different objects. Results show that VRHapticDrones is best suited to simulate objects that are expected to feel either light-weight or have yielding surfaces. With VRHapticDrones we contribute a solution to provide unintrusive and flexible feedback as well as insights for future VR haptic feedback systems.

Recent advances in technology and miniaturization allow the building of self-levitating tangible interfaces. This includes flying tangibles, which extend the mid-air interaction space from 2D to 3D. While a number of theoretical concepts about interaction with levitating tangibles were previously investigated by various researchers, a user-centered evaluation of the presented interaction modalities has attracted only minor attention from prior research. We present Flyables, a system adjusting flying tangibles in 3D space to enable interaction between users and levitating tangibles. Interaction concepts were evaluated in a user study(N=17), using quadcopters as operable levitating tangibles. Three different interaction modalities are evaluated to collect quantitative data and qualitative feedback. Our findings show preferred user interaction modalities using Flyables. We conclude our work with a discussion and future research within the domain of human-drone interaction.

Manual assembly at production is a mentally demanding task. With rapid prototyping and smaller production lot sizes, this results in frequent changes of assembly instructions that have to be memorized by workers. Assistive systems compensate this increase in mental workload by providing "just-in-time" assembly instructions through in-situ projections. The implementation of such systems and their benefits to reducing mental workload have previously been justified with self-perceived ratings. However, there is no evidence by objective measures if mental workload is reduced by in-situ assistance. In our work, we showcase electroencephalography (EEG) as a complementary evaluation tool to assess cognitive workload placed by two different assistive systems in an assembly task, namely paper instructions and in-situ projections. We identified the individual EEG bandwidth that varied with changes in working memory load. We show, that changes in the EEG bandwidth are found between paper instructions and in-situ projections, indicating that they reduce working memory compared to paper instructions. Our work contributes by demonstrating how design claims of cognitive demand can be validated. Moreover, it directly evaluates the use of assistive systems for delivering context-aware information. We analyze the characteristics of EEG as real-time assessment for cognitive workload to provide insights regarding the mental demand placed by assistive systems.

Smartphones, wearables, and other mobile devices often use tactile feedback for notifying users. This feedback type proved to be beneficial since it does not occupy the visual or auditory channel. However, it still can be distracting in other situations such as when users are already stressed. To investigate tactile feedback patterns which do not increase the user's stress level, we developed two wrist-worn prototypes capable of providing tactile feedback (i.e., vibrotactile and pressure-based feedback). Further, we conducted a user-study with 14 participants comparing both feedback types. The results suggest that vibrotactile feedback increases the user's stress level more, compared to pressure-based feedback particularly applied when the user currently has a low stress level. Consequently, we present implications for designing notifications for mobile and wearable devices.

More and more industrial manufacturing companies are outsourcing assembly tasks to sheltered work organizations where cognitively impaired workers are employed. To facilitate these assembly tasks assistive systems have been introduced to provide cognitive assistance. While previous work found that these assistive systems have a great impact on the workers' performance in giving assembly instructions, these systems are further capable of detecting errors and notifying the worker of an assembly error. However, the topic of how assembly errors are presented to cognitively impaired workers has not been analyzed scientifically. In this paper, we close this gap by comparing tactile, auditory, and visual error feedback in a user study with 16 cognitively impaired workers. The results reveal that visual error feedback leads to a significantly faster assembly time compared to tactile error feedback. Further, we discuss design implications for providing error feedback for workers with cognitive impairments.

With increasing complexity of assembly tasks and an increasing number of product variants, instruction systems providing cognitive support at the workplace are becoming more important. Different instruction systems for the workplace provide instructions on phones, tablets, and head-mounted displays (HMDs). Recently, many systems using in-situ projection for providing assembly instructions at the workplace have been proposed and became commercially available. Although comprehensive studies comparing HMD and tablet-based systems have been presented, in-situ projection has not been scientifically compared against state-of-the-art approaches yet. In this paper, we aim to close this gap by comparing HMD instructions, tablet instructions, and baseline paper instructions to in-situ projected instructions using an abstract Lego Duplo assembly task. Our results show that assembling parts is significantly faster using in-situ projection and locating positions is significantly slower using HMDs. Further, participants make less errors and have less perceived cognitive load using in-situ instructions compared to HMD instructions.

With the opportunity to customize ordered products, assembly tasks are becoming more and more complex. To meet these increased demands, a variety of interactive instruction systems have been introduced. Although these systems may have a big impact on overall efficiency and cost of the manufacturing process, it has been difficult to optimize them in a scientific way. The challenge is to introduce performance metrics that apply across different tasks and find a uniform experiment design. In this paper, we address this challenge by proposing a standardized experiment design for evaluating interactive instructions and making them comparable with each other. Further, we introduce a General Assembly Task Model, which differentiates between task-dependent and task-independent measures. Through a user study with 12 participants, we evaluate the experiment design and the proposed task model using an abstract pick-and-place task and an artificial industrial task. Finally, we provide paper-based instructions for the proposed task as a baseline for evaluating Augmented Reality instructions.