Translating physical games to digital tabletops is a promising way to engage museum visitors in open-ended museum experiences, but the process is poorly understood. How can the mechanics and social dynamics of physical games be translated into this new material form? This paper presents an empirical account of translating a physical, history-themed card game to a tangible tabletop for museum play. Through a year-long Research through Design study, we developed and deployed three comparative prototypes, each investigating a key dimension of translation: (1) the shift from turn-based to real-time event placement, (2) the introduction of time pressure, and (3) the adaptation of game difficulty. Drawing on in-situ playtests with families and workshops with expert designers, our findings reveal a central tension between fidelity to the source material’s social strengths and the transformative affordances of the digital medium. We contribute actionable design implications for translation and an open-source toolkit to support future work.

Large Language Models (LLMs) are poised to revolutionize interactive storytelling, yet they introduce a fundamental HCI challenge: balancing player agency with the coherence of a pre-authored narrative. Existing LLM-driven Game Masters (GMs) often undermine the player experience by being overly compliant, leading to disjointed stories, or by rigidly ‘railroading’ players, which diminishes their sense of freedom. This paper addresses this tension by introducing SENNA, a novel multi-agent AI system designed to maintain narrative adherence in single-player role-playing games. SENNA operationalizes a prewritten adventure by creating and using a Narrative Graph to model the structure of the story, track the player’s progress and ensure key plot points are met while maintaining interactivity. A primary contribution of this work is an empirical investigation of narrative redirection, the methods an AI GM can use to guide players back to the intended arc when they deviate. We designed six redirection strategies inspired by expert human GMs and conducted a user study to evaluate their effectiveness. The study combined live gameplay with a within-subjects comparison of different strategies at key narrative junctions. Our findings show that players strongly prefer redirection techniques grounded in the internal logic of the game, such as offering more information, experiencing consequences in the world, and influence from non-player characters. These methods successfully maintained narrative coherence without sacrificing perceived autonomy or immersion, while simplistic ‘hard denials’ were poorly received. Our work contributes an empirically validated framework and actionable design guidelines to create more robust and human-centered AI-driven narrative experiences.

Emotional voice conversion (EVC) focuses on converting a speech utterance from a source to a target emotion; it can thus be a key enabling technology for human-computer interaction applications and beyond. However, EVC remains an unsolved research problem with several challenges. In particular, as speech rate and rhythm are two key factors of emotional conversion, models have to generate output sequences of differing length. Sequence-to-sequence modelling is recently emerging as a competitive paradigm for models that can overcome those challenges. In an attempt to stimulate further research in this promising new direction, recent sequence-to-sequence EVC papers were systematically investigated and reviewed from six perspectives: their motivation, training strategies, model architectures, datasets, model inputs, and evaluation methods. This information is organised to provide the research community with an easily digestible overview of the current state-of-the-art. Finally, we discuss existing challenges of sequence-to-sequence EVC.

Gesture recognition on smartwatches is challenging not only due to resource constraints but also due to the dynamically changing conditions of users. It is currently an open problem how to engineer gesture recognisers that are robust and yet deployable on smartwatches. Recent research has found that common everyday events, such as a user removing and wearing their smartwatch again, can deteriorate recognition accuracy significantly. In this paper, we suggest that prior understanding of causes behind everyday variability and false positives should be exploited in the development of recognisers. To this end, first, we present a data collection method that aims at diversifying gesture data in a representative way, in which users are taken through experimental conditions that resemble known causes of variability (e.g., walking while gesturing) and are asked to produce deliberately varied, but realistic gestures. Secondly, we review known approaches in machine learning for recogniser design on constrained hardware. We propose convolution-based network variations for classifying raw sensor data, achieving greater than 98% accuracy reliably under both individual and situational variations where previous approaches have reported significant performance deterioration. This performance is achieved with a model that is two orders of magnitude less complex than previous state-of-the-art models. Our work suggests that deployable and robust recognition is feasible but requires systematic efforts in data collection and network design to address known causes of gesture variability.

A human's heart beating can be sensed by sensors and displayed for others to see, hear, feel, and potentially "resonate'' with. Previous work in studying interaction designs with physiological data, such as a heart's pulse rate, have argued that feeding it back to the users may, for example support users' mindfulness and self-awareness during various everyday activities and ultimately support their health and wellbeing. Inspired by Somaesthetics as a discipline, we designed and explored multimodal displays, which enable experiencing heart beats as natural stimuli from oneself and others in social proximity. In this paper, we report on the design process of our design PiHearts and present qualitative results of a field study with 30 pairs of participants. Participants were asked to use PiHearts during watching short movies together and report their perceived experience in three different display conditions while watching movies. We found, for example that participants reported significant effects in experiencing sensory immersion when they received their own heart beats as stimuli compared to the condition without any heart beat display, and that feeling their partner's heart beats resulted in significant effects on social experience. We refer to resonance theory to motivate and discuss the results, highlighting the potential of how digitalization of heart beats as rhythmic natural stimuli may provide resonance in a modern society facing social acceleration.

Non-verbal sounds are an essential communication channel for social robots. However, it requires expert knowledge to create and compose synthesizers, develop melodic structures or record samples which express a robot's internal intentions and emotions. This paper presents an approach for adapting a robot's timbre based on non-expert human comparative feedback in order to personalize the sonic interaction design to an individual user's preferences. An evolution strategy learns parameters of real-time sound synthesis for different intentions and emotions. Ultimately, the strategy aims to improve the perceived goodness of how well a specific melody's sound maps to a specific emotion or intention. In order to demonstrate the feasibility of the approach, we report on a user study with a robot, 6 exemplary melodies and 27 participants. Our study results show that the strategy indeed results in improved and preferred sound designs and that many participants are willing to apply such a process to improve their robots' expressivity.

Creativity as a skill is associated with a potential to drive both productivity and psychological wellbeing. Since multimodality can foster cognitive ability, multimodal digital tools should also be ideal to support creativity as an essentially cognitive skill. In this paper, we explore this notion by presenting a multimodal pen-based interaction technique and studying how it supports creativity. The multimodal solution uses micro-controller-technology to augment a digital pen with RGB LEDs and a Leap Motion sensor to enable bimanual input. We report on a user study with 26 participants demonstrating that the multimodal technique is indeed perceived as supporting creativity significantly more than a baseline condition.

Finger touch and mouse-based interaction are today's predominant modalities to interact with screen-based user interfaces. Related work suggests that new techniques interweaving pre-touch sensing and touch are useful future alternatives. In this paper, we introduce Fich, a novel pre-touch technique that augments conventional touch interfaces with tooltips and further "fingerover" effects, opening up the space in front of the screen for user interaction. To study Fich in-depth, we developed a Fich-enabled weather application and compared user experience and interface discovery ("serendipity") of Fich against the traditional input modalities Mouse and finger Touch in a user study with 42 subjects. We report on the results, implying Fich's user experience to be rated significantly higher in terms of hedonic quality and significantly lower in terms of pragmatic quality, as compared to traditional input modalities.

First impressions play an important part in social interactions, establishing the foundation of a person's opinion about their counterparts. Since interpersonal communication is essentially multimodal, people are judged during first encounters by both their verbal utterances and nonverbal behavior, such as how they utilize eye contact, body distance, and body orientation. In this paper, we argue that robots would provide better user experiences, including being perceived as more likable if they were able to make a good first impression when introduced to a new home. Moreover, we wanted to test if robots can improve their perceived impression by behaving in a proxemic-aware manner; i.e., by following established social norms, which prescribe, for example how far people should position themselves around other objects to improve the facilitation of social interactions. In order to test this hypothesis, we conducted a user study with 16 participants in a virtual reality setting, comparing the impression of two agents being introduced to their new homes by users. We found that the proxemic-aware agent was indeed perceived as significantly better considering multiple constructs, including perceived anthropomorphism and trustworthiness.

There seems to be a danger to carelessly replace routine tasks in homes through automation with IoT-technology. But since routines, such as watering houseplants also have positive influences on inhabitants' wellbeing, they should be transformed through carefully performed designs. To this end, an attempt to use technology for augmenting a set of houseplants' non-verbal communication capabilities is presented. First, we describe in detail how implicit interactions have been designed to support inhabitants in watering their plants through meaningful interactions. Then, we report on a field study with 24 participants, comparing two alternative design implementations based on contrasting embodied interaction technologies (i.e., augmented reality and embedded computing technology). The study results highlight shortcomings of today's smartphone mediated augmented reality compared to physical interface alternatives, considering measurements of perceived attractiveness and expected effects on determinants of wellbeing, and discusses potentials of combining both modalities for future solutions.

The user experience (UX) of graphical user interfaces (GUIs) often depends on how clearly visual designs communicate/signify "affordances", such as if an element on the screen can be pushed, dragged, or rotated. Especially for novice users figuring out the complexity of a new interface can be cumbersome. In the "past" era of mouse-based interaction mouseover effects were successfully utilized to trigger a variety of assistance, and help users in exploring interface elements without causing unintended interactions and associated negative experiences. Today's GUIs are increasingly designed for touch and lack a method similiar to mouseover to help (novice) users to get acquainted with interface elements. In order to address this issue, we have studied gazeover, as a technique for triggering "help or guidance" when a user's gaze is over an interactive element, which we believe is suitable for today's touch interfaces. We report on a user study comparing pragmatic and hedonic qualities of gazeover and mouseover, which showed significant higher ratings in hedonic quality for the gazeover technique. We conclude by discussing limitations and implications of our findings.

Combining mid-air gestures with pen input for bi-manual input on tablets has been reported as an alternative and attractive input technique in drawing applications. Previous work has also argued that mid-air gestural input can cause discomfort and arm fatigue over time, which can be addressed in a desktop setting by allowing users to gesture in alternative restful arm positions (e.g., elbow rests on desk). However, it is unclear if and how gesture preferences and gesture designs would be different for alternative arm positions. In order to inquire these research question we report on a user and choice based gesture elicitation study in which 10 participants designed gestures for different arm positions. We provide an in-depth qualitative analysis and detailed categorization of gestures, discussing commonalities and differences in the gesture sets based on a "think aloud" protocol, video recordings, and self-reports on user preferences.

A shared sense of humor can result in positive feelings associated with amusement, laughter, and moments of bonding. If robotic companions could acquire their human counterparts' sense of humor in an unobtrusive manner, they could improve their skills of engagement. In order to explore this assumption, we have developed a dynamic user modeling approach based on Reinforcement Learning, which allows a robot to analyze a person's reaction while it tells jokes and continuously adapts its sense of humor. We evaluated our approach in a test scenario with a Reeti robot acting as an entertainer and telling different types of jokes. The exemplary adaptation process is accomplished only by using the audience's vocal laughs and visual smiles, but no other form of explicit feedback. We report on results of a user study with 24 participants, comparing our approach to a baseline condition (with a non-learning version of the robot) and conclude by providing limitations and implications of our approach in detail.

Digital drawing experiences are not only fused by the flexibility of digital materials but also influenced by the availability of interaction space. In this paper, we first present a prototype, which implements a method to turn the (mid-air) space above and aside a drawing screen in a desktop setting dynamically into sensory space for gestural and spatial input. Then we report on a user study exploring how participants experience digital drawing when the additional interaction space above and aside a screen is exploited for exemplary proxemic input techniques for zooming and panning a drawing. Our results show that the new multimodal input techniques are perceived as significantly more attractive than a baseline drawing condition which only utilizes touch based input. We conclude by discussing implications and limitations of our findings and input above and aside a drawing screen in general.

The number of commuters has been increasing for many years and the negative effects on wellbeing are therefore affecting more and more people. Following a user centered design process that focuses on known wellbeing determinants, such as relatedness and empathy, we developed the Honeypot socializing app. The app allows commuters to find other travelers to chat with and meet in person to enhance their wellbeing through fostering meaningful and contextual social interactions. First, we describe the development of the idea and the design of the app. Then, we report on a field study with 16 participants, which we carried out on trains. The study results show that the app helps to get in contact with fellow travelers and that it has the potential to promote the wellbeing of commuters in the long term.

Proxemic touch targets continuously change in relation to a user's hand in mid-air before a physical touch occurs. Previous work has, for example shown that expanding targets are capable to improve target acquisition performance on touch interfaces. However, it is unclear how proxemic touch targets influence user experience (UX) in a broader sense, including hedonic qualities. Towards closing this research gap the paper reports on two user studies. The first study is a qualitative study with five experts, providing in-depth insights on a variety of change-types (e.g., size, form, color) and how they, for example, influence perceived functional and aesthetic qualities of proxemic touch targets. A follow-up user study with 36 participants explores the UX of a proxemic touch target compared to non-proximal versions of the same target. The results highlight a positive significant effect of the proxemic design on both pragmatic and hedonic qualities.

Combining touch screen technology with mid-air gestures into a unified input modality has potential to improve interaction with touch interfaces in cars. Moreover, target objects on a touch screen can be adapted to the proximity of a users' finger in mid-air. In this paper, we present an exploration of this design space based on two studies and various prototypical systems. First, results of a driving simulator study are reported, indicating that a driver's performance in acquiring a target object on a touch screen in a central console position increases with expanding targets, while altering the position of a target object on the screen leads to a decrease of performance. In a follow-up workshop with automotive experts, prototypes with multiple expanding targets were utilized to foster in-depth discussions on potential challenges and benefits with expanding targets in cars. Results of both studies indicate a high potential of expanding targets for in-car interaction scenarios.

During conversational practices, such as a tablet-mediated sales conversation between a salesperson and a customer, tablets are often used by two users who prefer specific bodily formations in order to easily face each other and the surface of the touchscreen. In a series of studies, we investigated bodily formations that are preferred during tablet-mediated sales conversations, and explored the effect of these formations on performance in acquiring touch targets (e.g., buttons) on a tablet device. We found that bodily formations cause decreased viewing angles to the shared screen, which results in a decreased performance in target acquisition. In order to address this issue, a multi-modal design consideration is presented, which combines mid-air finger movement and touch into a unified input modality, allowing the design of proximity sensitive touch targets. We conclude that the proposed embodied interaction design not only has potential to improve targeting performance, but also adapts the ``agency' of touch targets for multi-user settings.

Theories of how the human mind works have inspired but also constrained negatively and positively - interaction design. Today's interfaces can potentially take any (physical and digital) form. Therefore, there is a need to prevent users from learning necessities for every single interface. To address the issue, interaction designers increasingly constrain their design decisions based on their conceptions of users' capacities and skills that already exist and are based on prior interactions with the real world. These skills, such as hand-eye coordination, are essential for tangible interfaces, and directly relate to cognition. In this one-day workshop, we will discuss the role of theories of mind in contemporary human-computer interaction research, as those might provide us with a substantial understanding of cognition and human skills. The focus will be on embodied and situated theories and possible consequences of adopting an embodied and situated perspective on the design and analysis of tangible and embedded interaction.

Handheld touch screen based devices, such as smartphones or tablets, typically provide limited haptic feedback. On cur- rent devices, what is visually perceived and what is tactile and kinesthetically felt is semantically uncoupled. In order to improve embodied interaction on handheld screen based devices, new ways to provide richer haptic feedback are re- quired. In this paper, we present TorqueScreen, a prototypi- cal system combining a handheld touch screen device (i.e., a tablet) with an actuated flywheel capable of imposing angular momentum onto the tablet. The TorqueScreen design allows interlinking the movement and physics of virtual objects on the screen with the torque as haptic output imposed on the tablet. We describe the design and implementation of the de- vice, illustrate the resulting design space, and discuss future improvements and potential applications.

Authentication based on touch-less mid-air gestures would benefit a multitude of ubicomp applications, which are used in clean environments (e.g., medical environments or clean rooms). In order to explore the potential of mid-air gestures for novel authentication approaches, we performed a series of studies and design experiments. First, we collected data from more then 200 users during a three-day science event organised within a shopping mall. This data was used to investigate capabilities of the Leap Motion sensor and to formulate an initial design problem. The design problem, as well as the design of mid-air gestures for authentication purposes, were iterated in subsequent design activities. In a final study with 13 participants, we evaluated two mid-air gestures for authentication purposes in different situations, including different body positions. Our results highlight a need for different mid-air gestures for differing situations and carefully chosen constraints for mid-air gestures.

In this paper we report a user study to investigate the effect of semantic weight in a touch-based drag and drop task. The study was motivated by our own interest in exploring potential factors that influence touch behavior and supported by results in related neuroscience research. The question we intended to answer is: "Do people drag the representation of a smaller and lighter real world object (e.g. an apple) different than the representation of a heavier and larger real world object (e.g. a car)?". Participants were asked to perform a drag and drop task repeatedly on a tablet device. Dragged objects were the same physical size on screen, but represented real world objects that were either heavy and large or light and small. We studied two representation modalities (i.e. image and text). In both representation modalities, semantically heavier objects were dragged significantly faster than semantically lighter objects.

In this paper we describe an exploratory design study of the potentials of taste as a playful interaction modality. We present the design and implementation of LOLLio -- an interactive lollipop that serves as a haptic input device that dynamically changes its taste. We conclude this paper with three basic principles for potential game designs, where we see how the interactive lollipop we have built can foster novel, playful game experiences.

S.499-502

Normally, mobile pedestrian navigation systems use visually perceptible landmarks to guide their users through the environment. In this article we introduce concepts for the use of auditory perceptible landmarks in route descriptions. Such auditory perceptible landmarks complement their visual counterparts and also stand to be beneficial for certain groups like the visually impaired and the elderly.

In this paper we regard the navigation aid provided by mobile navigation systems in a real environment and the effects of these mobile assistants to the development of spatial knowledge. Therefore, we report on a user study concerning the acquisition of spatial knowledge. This study sets up on a former study described by Krüger and colleagues and sheds light on problems concerning the acquisition of survey knowledge while being navigated by a mobile handheld PC.