Conventional navigation systems, fixated on metrics such as time and distance, neglect the driver’s emotional well-being, despite driving routes being inherent emotional triggers. This raises a critical question for the Intelligent User Interface community: How can intelligent systems successfully route information based on emotion? To address this gap, we introduce HappyRouting, an empathic car interface designed as an initial attempt to guide drivers through real-world traffic while actively optimizing for positive emotional states. Our core technical contribution is a machine learning-based emotion map layer that predicts the affective valence along various routes using both static and dynamic contextual data. HappyRouting enables the generation of “happy routes” integrated into a functional vehicular interface prototype. We explored the efficacy of this approach in a preliminary, small-scale driving study (N = 13). Our initial findings provide provocative evidence: Emotion-optimized routes successfully increased the subjectively perceived valence by 11% (p =.007) compared to standard routes. Furthermore, despite taking 1.25 times longer on average, participants consistently perceived the travel duration as shorter. This result suggests that integrating emotional optimization could fundamentally challenge the speed-first paradigm. However, recognizing the constraints of our initial, limited sample, we conclude by discussing ethical and computational challenges that must be resolved before emotion-based routing can be safely and scalably integrated into next-generation intelligent navigation apps.

Device orchestration in ubiquitous computing remains cumbersome and highly static, particularly given the growing number of personal devices. Routine tasks, such as preparing to read at home, often require multiple manual interactions (lighting, heating, music), which are difficult to automate with rule-based systems due to contextual sensitivity. We present GenX, a perception-driven approach that applies multimodal generative AI to create dynamic, adaptive user experiences. By leveraging vision-language models, GenX interprets a user’s perception of their environment and generates a multimodal UX tailored to available output modalities. The system produces and executes source code in real time, enabling devices to be orchestrated flexibly and contextually. We introduce the concept of Generative Experiences as a scalable solution for dynamic UX design, capable of handling unexpected situations. GenX is demonstrated in two in-the-wild scenarios and a user study, showing positive user perceptions and effectiveness in supporting everyday tasks.

Stories have long captivated the human imagination with narratives that enrich our lives. Traditional storytelling methods are often static and not designed to adapt to the listener’s environment, which is full of dynamic changes. For instance, people often listen to stories in the form of podcasts or audiobooks while traveling in a car. Yet, conventional in-car storytelling systems do not embrace the adaptive potential of this space. The advent of generative AI is the key to creating content that is not just personalized but also responsive to the changing parameters of the environment. We introduce a novel system for interactive, real-time story narration that leverages environment and user context in correspondence with estimated arrival times to adjust the generated story continuously. Through two comprehensive real-world studies with a total of 30 participants in a vehicle, we assess the user experience, level of immersion, and perception of the environment provided by the prototype. Participants’ feedback shows a significant improvement over traditional storytelling and highlights the importance of context information for generative storytelling systems.

Deep learning based electroencephalography (EEG) signal processing methods are known to suffer from poor test-time generalization due to the changes in data distribution. This becomes a more challenging problem when privacy-preserving representation learning is of interest such as in clinical settings. To that end, we propose a multi-source learning architecture where we extract domain-invariant representations from dataset-specific private encoders. Our model utilizes a maximum-mean-discrepancy (MMD) based domain alignment approach to impose domain-invariance for encoded representations, which outperforms state-of-the-art approaches in EEG-based emotion classification. Furthermore, representations learned in our pipeline preserve domain privacy as dataset-specific private encoding alleviates the need for conventional, centralized EEG-based deep neural network training approaches with shared parameters.

There is a growing need for sparse representational formats of human affective states that can be utilized in scenarios with limited computational memory resources. We explore whether representing neural data, in response to emotional stimuli, in a latent vector space can serve to both predict emotional states as well as generate synthetic EEG data that are participant-and/or emotion-specific. We propose a conditional variational autoencoder based framework, EEG2Vec, to learn generative-discriminative representations from EEG data. Experimental results on affective EEG recording datasets demonstrate that our model is suitable for unsupervised EEG modeling, classification of three distinct emotion categories (positive, neutral, negative) based on the latent representation achieves a robust performance of 68.49%, and generated synthetic EEG sequences resemble real EEG data inputs to particularly reconstruct low-frequency signal components. Our work advances areas where affective EEG representations can be useful in e.g., generating artificial (labeled) training data or alleviating manual feature extraction, and provide efficiency for memory constrained edge computing applications.

Despite the advances in machine perception, semantic scene understanding is still a limiting factor in mixed reality scene composition. In this paper, we present TransforMR, a video see-through mixed reality system for mobile devices that performs 3D-pose-aware object substitution to create meaningful mixed reality scenes. In real-time and for previously unseen and unprepared real-world environments, TransforMR composes mixed reality scenes so that virtual objects assume behavioral and environment-contextual properties of replaced real-world objects. This yields meaningful, coherent, and humaninterpretable scenes, not yet demonstrated by today’s augmentation techniques. TransforMR creates these experiences through our novel pose-aware object substitution method building on different 3D object pose estimators, instance segmentation, video inpainting, and pose-aware object rendering. TransforMR is designed for use in the real-world, supporting the substitution of humans and vehicles in everyday scenes, and runs on mobile devices using just their monocular RGB camera feed as input. We evaluated TransforMR with eight participants in an uncontrolled city environment employing different transformation themes. Applications of TransforMR include real-time character animation analogous to motion capturing in professional film making, however without the need for preparation of either the scene or the actor, as well as narrative-driven experiences that allow users to explore fictional parallel universes in mixed reality. We make all of our source code and assets available1.1TransforMR code release: https://github.com/MohamedKari/transformr

Detecting emotions while driving remains a challenge in Human-Computer Interaction. Current methods to estimate the driver’s experienced emotions use physiological sensing (e.g., skin-conductance, electroencephalography), speech, or facial expressions. However, drivers need to use wearable devices, perform explicit voice interaction, or require robust facial expressiveness. We present VEmotion (Virtual Emotion Sensor), a novel method to predict driver emotions in an unobtrusive way using contextual smartphone data. VEmotion analyzes information including traffic dynamics, environmental factors, in-vehicle context, and road characteristics to implicitly classify driver emotions. We demonstrate the applicability in a real-world driving study (N = 12) to evaluate the emotion prediction performance. Our results show that VEmotion outperforms facial expressions by 29% in a person-dependent classification and by 8.5% in a person-independent classification. We discuss how VEmotion enables empathic car interfaces to sense the driver’s emotions and will provide in-situ interface adaptations on-the-go.

Music is a central instrument in video gaming to attune a player’s attention to the current atmosphere and increase their immersion in the game. We transfer the idea of scene-adaptive music to car drives and propose SoundsRide, an in-car audio augmented reality system that mixes music in real-time synchronized with sound affordances along the ride. After exploring the design space of affordance-synchronized music, we design SoundsRide to temporally and spatially align high-contrast events on the route, e. g., highway entrances or tunnel exits, with high-contrast events in music, e. g., song transitions or beat drops, for any recorded and annotated GPS trajectory by a three-step procedure. In real-time, SoundsRide 1) estimates temporal distances to events on the route, 2) fuses these novel estimates with previous estimates in a cost-aware music-mixing plan, and 3) if necessary, re-computes an updated mix to be propagated to the audio output. To minimize user-noticeable updates to the mix, SoundsRide fuses new distance information with a filtering procedure that chooses the best updating strategy given the last music-mixing plan, the novel distance estimations, and the system parameterization. We technically evaluate SoundsRide and conduct a user evaluation with 8 participants to gain insights into how users perceive SoundsRide in terms of mixing, affordances, and synchronicity. We find that SoundsRide can create captivating music experiences and positively as well as negatively influence subjectively perceived driving safety, depending on the mix and user.

For more than two decades, capacitive sensing has played a prominent role in human-computer interaction research. Capacitive sensing has become ubiquitous on mobile, wearable, and stationary devices - enabling fundamentally new interaction techniques on, above, and around them. The research community has also enabled human position estimation and whole-body gestural interaction in instrumented environments. However, the broad field of capacitive sensing research has become fragmented by different approaches and terminology used across the various domains. This paper strives to unify the field by advocating consistent terminology and proposing a new taxonomy to classify capacitive sensing approaches. Our extensive survey provides an analysis and review of past research and identifies challenges for future work. We aim to create a common understanding within the field of human-computer interaction, for researchers and practitioners alike, and to stimulate and facilitate future research in capacitive sensing.

Platypus is the first system to localize and identify people by remotely and passively sensing changes in their body electric potential which occur naturally during walking. While it uses three or more electric potential sensors with a maximum range of 2 m, as a tag-free system it does not require the user to carry any special hardware. We describe the physical principles behind body electric potential changes, and a predictive mathematical model of how this affects a passive electric field sensor. By inverting this model and combining data from sensors, we infer a method for localizing people and experimentally demonstrate a median localization error of 0.16 m. We also use the model to remotely infer the change in body electric potential with a mean error of 8.8 % compared to direct contact-based measurements. We show how the reconstructed body electric potential differs from person to person and thereby how to perform identification. Based on short walking sequences of 5 s, we identify four users with an accuracy of 94 %, and 30 users with an accuracy of 75 %. We demonstrate that identification features are valid over multiple days, though change with footwear.

We explore the design space of energy-neutral situated displays, which give physical presence to digital information. We investigate three central dimensions: energy sources, display technologies, and wireless communications. Based on the power implications from our analysis, we present a thin, wireless, photovoltaic-powered display that is quick and easy to deploy and capable of indefinite operation in indoor lighting conditions. The display uses a low-resolution e-paper architecture, which is 35 times more energy-efficient than smaller-sized high-resolution displays. We present a detailed analysis on power consumption, photovoltaic energy harvesting performance, and a detailed comparison to other display-driving architectures. Depending on the ambient lighting, the display can trigger an update every 1 -- 25 minutes and communicate to a PC or smartphone via Bluetooth Low-Energy.

The increasing number of smart objects in our everyday life shapes how we interact beyond the desktop. In this workshop we discussed how the interaction with these smart objects should be designed from various perspectives. This year's workshop put a special focus on affective computing with smart objects, as reflected by the keynote talk.

Smart objects within instrumented environments offer an always available and intuitive way of interacting with a system. Connecting these objects to other objects in range or even to smartphones and computers, enables substantially innovative interaction and sensing approaches. In this paper, we investigate the concept of Capacitive Near-Field Communication to enable ubiquitous interaction with everyday objects in a short-range spatial context. Our central contribution is a generic framework describing and evaluating this communication method in Ubiquitous Computing. We prove the relevance of our approach by an open-source implementation of a low-cost object tag and a transceiver offering a high-quality communication link at typical distances up to 15 cm. Moreover, we present three case studies considering tangible interaction for the visually impaired, natural interaction with everyday objects, and sleeping behavior analysis.

Swiss-Cheese Extended proposes a novel real-time method for recognizing objects with capacitive proximity sensors. Applying this technique to ubiquitous user interfaces, it is possible to detect the 3D-position of multiple human hands in different configurations above a surface that is equipped with a small number of sensors. The retrieved object configurations can significantly improve a user's interaction experience or an application's execution context, for example by detecting multi-hand zoom and rotation gestures or recognizing a grasping hand. We emphasize the broad applicability of the proposed method with a study of a multi-hand gesture recognition device.

Capacitive sensing allows the creation of unobtrusive user interfaces that are based on measuring the proximity to objects or recognizing their dielectric properties. Combining the data of many sensors, applications such as in-the-air gesture recognition, location tracking or fluid-level sensing can be realized. We present OpenCapSense, a highly flexible open-source toolkit that enables researchers to implement new types of pervasive user interfaces with low effort. The toolkit offers a high temporal resolution with sensor update rates up to 1 kHz. The typical spatial resolution varies between one millimeter at close object proximity and around one centimeter at distances of 35cm or above.