Translating physical games to digital tabletops is a promising way to engage museum visitors in open-ended museum experiences, but the process is poorly understood. How can the mechanics and social dynamics of physical games be translated into this new material form? This paper presents an empirical account of translating a physical, history-themed card game to a tangible tabletop for museum play. Through a year-long Research through Design study, we developed and deployed three comparative prototypes, each investigating a key dimension of translation: (1) the shift from turn-based to real-time event placement, (2) the introduction of time pressure, and (3) the adaptation of game difficulty. Drawing on in-situ playtests with families and workshops with expert designers, our findings reveal a central tension between fidelity to the source material’s social strengths and the transformative affordances of the digital medium. We contribute actionable design implications for translation and an open-source toolkit to support future work.

Ensuring the safety of personnel at large industrial sites is a significant challenge. Misunderstandings and increased safety risks are related to communication, as traditional non-targeted alerts like sirens are often ineffective. To address this, we propose a drone-based system that uses voice interaction to deliver timely and targeted safety advisories. This paper presents a multi-stage, user-centered study, beginning with formative interviews with security and safety managers and culminating in an on-site evaluative study with personnel at two biomass power plants. Through our study, we identify key design insights for creating a drone that is perceived as an official, authoritative, and non-threatening entity. We provide design guidance for using sound and movement to signal intent, creating a voice persona that balances authority with clarity, and designing interaction flows to manage confusion and non-compliance. The results demonstrate the system's potential not only to enhance safety compliance, but also to mitigate interpersonal confrontation. First, we provide an empirically-grounded understanding of contextual safety challenges, which informs a set of design principles for intelligible and socially acceptable voice advisories from drones. Second, we present results from an in-situ evaluation of a working prototype that demonstrates its effectiveness in a real-world setting.

Patrolling tasks in multi-robot systems are essential for applications such as surveillance and monitoring, where minimizing the time between visits to any given location is critical. This paper investigates fault-tolerant redistribution strategies for multi-agent patrolling systems in partitioned environments using Unmanned Autonomous Vehicles (UAVs). We propose Heuristic Meeting-based Patrolling (HMP), a novel distributed and fault-tolerant patrolling algorithm. Building on the Heuristic Conscientious Reactive (HCR) strategy and incorporating periodic synchronization meetings, HMP enables decentralized coordination and dynamic fault recovery through minimal communication. UAVs exchange information at shared meeting points, enabling detection of failures and redistribution of responsibilities. We evaluate HMP and its simplified variants in various simulated environments using the Multi-Agent Exploration and Patrolling Simulator (MAEPS). The results demonstrate that HMP offers strong performance under both normal and fault conditions, comparative to state-of-the-art patrolling strategies in terms of idleness metrics. However, we also identify limitations in meeting scheduling under certain fault conditions, which can cause cascading failures. Based on these findings, we discuss potential improvements for future work, including enhanced meeting scheduling and adaptive partitioning strategies.

Large Language Models (LLMs) are poised to revolutionize interactive storytelling, yet they introduce a fundamental HCI challenge: balancing player agency with the coherence of a pre-authored narrative. Existing LLM-driven Game Masters (GMs) often undermine the player experience by being overly compliant, leading to disjointed stories, or by rigidly ‘railroading’ players, which diminishes their sense of freedom. This paper addresses this tension by introducing SENNA, a novel multi-agent AI system designed to maintain narrative adherence in single-player role-playing games. SENNA operationalizes a prewritten adventure by creating and using a Narrative Graph to model the structure of the story, track the player’s progress and ensure key plot points are met while maintaining interactivity. A primary contribution of this work is an empirical investigation of narrative redirection, the methods an AI GM can use to guide players back to the intended arc when they deviate. We designed six redirection strategies inspired by expert human GMs and conducted a user study to evaluate their effectiveness. The study combined live gameplay with a within-subjects comparison of different strategies at key narrative junctions. Our findings show that players strongly prefer redirection techniques grounded in the internal logic of the game, such as offering more information, experiencing consequences in the world, and influence from non-player characters. These methods successfully maintained narrative coherence without sacrificing perceived autonomy or immersion, while simplistic ‘hard denials’ were poorly received. Our work contributes an empirically validated framework and actionable design guidelines to create more robust and human-centered AI-driven narrative experiences.

In this paper, we investigate the use of shape-change for interaction with sound zones. A core challenge to designing interaction with sound zone systems is to support users’ understanding of the unique spatial properties of sound zones. Shape-changing interfaces present new opportunities for addressing this. We present a structured investigation into this. We leveraged the knowledge of 12 sound experts to define a set of basic shapes and movements. Then, we constructed a prototype and conducted an elicitation study with 17 novice users, investigating the experience of these shapes and movements. Our findings show that physical visualizations of sound zones can be useful in supporting users’ experience of sound zones. We present a framework of 4 basic pattern categories that prompt different sound zone experiences and outline further research directions for shape-change in supporting sound zone interaction.

Tangible input has been explored as a means for participants to self-report experiences while minimising disruption and allowing for discrete data collection. However, the accuracy of these tangible devices has not been studied systematically. We compared six input techniques, including slider, slider with resistance, capacitive touch slider, squeeze, rotary knob, and joystick, to understand their accuracy and resolution profile. Each of these wireless devices was designed in a similar form factor and intended to be operated discretely with one hand. We assessed input accuracy and participant perceptions across devices through a controlled lab study (N = 20), highlighting diverging limits to the accuracy of the input technique and possible explanations for the differences in resolution. Our results indicate that participant accuracy was highest using a slider, and lowest using a squeeze-based input. We discuss the suitability and challenges of discreet tangible self-report techniques, and highlight open research questions for future work.

Well-funded classrooms often provide a variety of learning materials such as computers, robotics and other expensive equipment to facilitate STEM Learning activities for children. However, exploring natural phenomena such as electrical energy is possible through very simple activities. In this paper we explore how cheap low-cost materials and simple electrochromic displays can be designed to support experiential learning about energy and power generation. For this we employed a mix of open-ended play and exploration as well as staged and goal-oriented activities. We developed two learning activities that involve children constructing working models that generate power including constructing a wind turbine and assembling a solar power harvesting house. We studied how children engaged in the activities and how the materials helped them understand the topic. All children could construct and complete the building tasks and were generally positive about the experience. We identified challenges encountered by children including interactions with the construction materials and electrochromic screens as well as insights about the mental models children have. We discuss challenges for staging learning through play with found low cost and local materials and provide implications for the design of constructionist oriented STEM learning.

Tabletop interactions can support various social activities such as collaborative review of dynamic sales information in the business setting or route planning with an interactive map among others. Sharing control between users of interactive surfaces can be difficult to ensure participation and efficiency for co-located contexts. We explore the differences between two modalities for interacting with a tabletop map including interactions with a voice assistant and tangible interactions with a shape-changing controller. To simulate a shared goal-oriented situation, we use the context of shared autonomous mobility and compare the tangible controller with voice input through simulated tasks of route planning and detours. The findings suggest that the most suitable interaction modality depends on user preferences, scenarios, and contexts. Tangible controls work well for communicating the state of the system and facilitating discussion, while voice can be more pragmatic and task oriented.

An increasing number of technologies are battling for our attention constantly throughout the day. The majority of distractions come in the form of notifications from mobile instant messaging services, which under certain circumstances can make us feel overwhelmed and stressed. The goal of this paper is to add to the growing body of research that aims to find solutions and coping mechanisms for the problems associated with excessive notifications. Instead of focusing on software solutions on the mobile device itself, we explore the use of multisensory ambient information displays as notification delivery systems in the home context. Our goal was to use the attention periphery to create sufficient awareness of incoming messages without overwhelming the user. We conducted a three-part investigation that began with a laboratory study with 23 participants exploring initial impressions with notifications across a variety of sensory channels and modalities. Insights from this study were then used in a design workshop that helped us develop four different ambient notification prototypes. We present qualitative findings from a subsequent study in which seven participants used those devices for a period of four days in their homes. Finally, we compare the results of different notification modalities and provide implications for the design of ambient multisensory notifications in the home context.

In this paper, we present a case study that explores how children could learn to interact with programmable matter. Flying drone swarms enable physical visualizations of complex data and simulation of physical objects and processes, e.g., planetary movements. The swarms can be digitally controlled as an ensemble as a form of (sparse) "programmable matter." We worked with the toy company LEGO®, to design and evaluate a "build and fly" experience with 240 children in a public exhibition. The children decorated a bendable handheld controller with LEGO® bricks and then used this controller to animate the flight of a 10-drone swarm. Results indicate that children enjoyed the constructive play and performance aspects of the system. Four main patterns of player behavior emerged, which we discuss in relation to possible improvements to the system. We provide implications for design of programmable matter systems for supporting child play experiences.

LightBee is a novel "hologrammatic" telepresence system featuring a self-levitating light field display. It consists of a drone that flies a projection of a remote user's head through 3D space. The movements of the drone are controlled by the remote user's head movements, offering unique support for non-verbal cues, especially physical proxemics. The light field display is created by a retro-reflective sheet that is mounted on the cylindrical quadcopter. 45 smart projectors, one per 1.3 degrees, are mounted in a ring, each projecting a video stream rendered from a unique perspective onto the retroreflector. This creates a light field that naturally provides motion parallax and stereoscopy without requiring any headset nor stereo glasses. LightBee allows multiple local users to experience their own unique and correct perspective of the remote user's head. The system is currently one-directional: 2 small cameras mounted on the drone allow the remote user to observe the local scene.

Shape-changing interfaces have emerged as a new method for interacting with computers, using dynamic changes in a device's physical shape for input and output. With the advances of research into shape-changing interfaces, we see a need to synthesize the main, open research questions. The purpose of this synthesis is to formulate common challenges across the diverse fields engaged in shape-change research, to facilitate progression from single prototypes and individual design explorations to grander scientific goals, and to draw attention to challenges that come with maturity, including those concerning ethics, theory-building, and societal impact. In this article we therefore present 12 grand challenges for research on shape-changing interfaces, derived from a three-day workshop with 25 shape-changing interface experts with backgrounds in design, computer science, human-computer interaction, engineering, robotics, and material science.

For telepresence to support the richness of multiparty conversations, it is important to convey motion parallax and stereoscopy without head-worn apparatus. TeleHuman2 is a "hologrammatic" telepresence system that conveys full-body 3D video of interlocutors using a human-sized cylindrical light field display. For rendering, the system uses an array of projectors mounted above the heads of participants in a ring around a retroreflective cylinder. Unique angular renditions are calculated from streaming depth video captured at the remote location. Projected images are retro-reflected into the eyes of local participants, at 1.3º intervals providing angular renditions simultaneously for left and right eyes of all onlookers, which conveys motion parallax and stereoscopy without head-worn apparatus or head tracking. Our technical evaluation of the angular accuracy of the system demonstrates that the error in judging the angle of a remote arrow object represented in TeleHuman2 is within 1 degree, and not significantly different from similar judgments of a collocated arrow object.

We present GridDrones, a self-levitating programmable matter platform that can be used for representing 2.5D voxel grid relief maps capable of rendering unsupported structures and 3D transformations. GridDrones consists of cube-shaped nanocopters that can be placed in a volumetric 1xnxn mid-air grid, which is demonstrated here with 15 voxels. The number of voxels and scale is only limited by the size of the room and budget. Grid deformations can be applied interactively to this voxel lattice by manually selecting a set of voxels, then assigning a continuous topological relationship between voxel sets that determines how voxels move in relation to each other and manually drawing out selected voxels from the lattice structure. Using this simple technique, it is possible to create unsupported structures that can be translated and oriented freely in 3D. Shape transformations can also be recorded to allow for simple physical shape morphing animations. This work extends previous work on selection and editing techniques for 3D user interfaces.

Blood sampling is a common and necessary procedure in the treatment and diagnosis of a variety of diseases. However, it often results in painful and stressful experiences for children. Designed together with domain experts, ChillFish is a breath-controlled biofeedback game technology with bespoke airflow sensor that aims to calm children during blood sampling procedures. An experimental pilot study was conducted in which 20 children aged 6-11 were assigned to one of two conditions involving either passive distraction (watching a video) or active distraction using the ChillFish prototype. Medical staff rated ChillFish significantly more useful in facilitating the blood sampling procedure compared to passive distraction. Qualitative feedback from patients, parents, and medical staff identified aspects that impact the acceptance of breath-based active distraction. Our study highlights the potential of non-pharmacological assistive technology tools to reduce fear and pain for children undergoing painful or stressful medical treatment.

Despite an increasing number of examples of shape-changing interfaces, the relation between users' actions and product movements has not gained a great deal of attention, nor been very well articulated. This paper presents a framework articulating the level of control offered to the user over the shape change. The framework considers whether the shape change is: 1) directly controlled by the user's explicit interactions; 2) negotiated with the user; 3) indirectly controlled by the users actions; 4) fully controlled by the system. The four types are described through design examples using ReFlex, a shape-changing interface in the form of a smartphone. The paper concludes that shape-changing interfaces tend to assign the control to either the user or the underlying system, while few (e.g. [16,28]) consider sharing the control between the user and the system.

While there has been much focus on tangible lighting interfaces embedded in physical objects and smartphones as remote control, there has not been sufficient attention on how the expressivity of bodily movement can be used when designing interactions with light. Therefore, we investigate interaction with lighting technology beyond the smartphone and physical controllers. We examine the usefulness of the in-air gestural interaction style for lighting control. We bring forward "Tangible Lights", which serves as a novel interface for in-air interaction with lighting, drawing on existing knowledge from the tangible world. Tangible Lights has been subject to initial evaluations.

Previous research on conversational, competitive, and cooperative systems suggests that people respond differently to humans and AI agents in terms of perception and evaluation of observed team-mate behavior. However, there has not been research examining the relationship between participants' protective behavior toward human/AI team-mates and their beliefs about their behavior. A study was conducted in which 32 participants played two sessions of a cooperative game, once with a "presumed" human and once with an AI team-mate; players could "draw fire" from a common enemy by "yelling" at it. Overwhelmingly, players claimed they "drew fire" on behalf of the presumed human more than for the AI team-mate; logged data indicates the opposite. The main contribution of this paper is to provide evidence of the mismatch in player beliefs about their actions and actual behavior with humans or agents and provides possible explanations for the differences.

In cooperative games that involve team-mates that are controlled by either a computer or another human player, is there a difference in how humans assign credit or blame? There has been some related work on computers as team-mates and credit/blame assignment, but there does not seem to have been work to show whether the belief that a team-mate is human or not affects this. A qualitative study was conducted, in which 16 participants played variations of a team-based game with one of four kinds of team-mates: human (real or perceived) or AI (real or perceived). The two main findings of this research are that the perception of whether a team-mate is human or computer results in different credit/blame assignment and results in inaccurate skill assessment.

Although there has been research suggesting that people will treat computers socially and even consider computers as team-mates, there does not seem to have been any research looking specifically at how the perception of team-mate identity affects game enjoyment and team-mate preference in real-time cooperative games. In order to study this question, a quantitative study was conducted in which 40 participants played a real-time, goal-oriented, cooperative game. During the study, all participants played the game twice: once with an AI team-mate and once with a "presumed" human team-mate (i.e., an AI team-mate that they believed was a human team-mate). Thus, the team-mate performance and behaviors were essentially the same for both conditions. Participants in the study showed a much higher preference for the "presumed" human team-mate, indicating significantly higher levels of enjoyment and cooperation during the game sessions with that team-mate. The results suggests that perceived identity is a strong moderator of game enjoyment.

Present computer games and digital entertainment do not usually facilitate intergenerational family interactions. According to recent survey results in Japan, there is a high percentage of older people who own and play electronic or computer games, but rarely do they play the games with their family members. It is a positive sign that more older people are participating in the digital games arena, but it would be even more beneficial if they could interact actively with the young family members through gaming activities. This could possibly strengthen family bonding and bridge the gap between older people and youth culture. This paper presents steps for designing an intergenerational family entertainment system which focuses on physical and social interactions using a mixed reality floor system. The main design goals include: facilitating interactions between users with varied levels of skill in utilizing technology, utilizing the familiar physical motions from other activities to make an intuitive physical interface, and encouraging social interactions among families and friends. Detailed implementation of these steps is presented in the design of our intergenerational entertainment system, Age Invaders. Four main prototype iterations for the system is presented. Our design process is based on User Centered Design and relies on constant involvement of users to understand the key issues and to help make effective design decisions. The results of the study help to focus the refinements of the existing platform from a usability standpoint and also aids in the development of new physical entertainment and interactive applications. This study provides insights into user issues including how users interact in a complex mixed reality experience, which is heavily based in physicality. The use of one portion of the user base which is most proficient with technology before involving the novice users was found to empower novice users to begin to use digital technology.

We describe a new form of interactive living media used to communicate social or ecological information in the form of an empathetic ambient media. In the fast paced modern world people are generally too busy to monitor various significant social or human aspects of their lives, such as time spent with their family, their overall health, state of the ecology, etc. By quantifying such information digitally, information is semantically coupled into living microorganisms, E. coli. Through the use of transformed DNA, the E. coli will then glow or dim according to the data. The core technical innovation of this system is the development of an information system based on a closed-loop control system through which digital input is able to control input fluids to the E. coli, and thereby control the output glow of the E. coli in real time. Thus, social or ecological based information is coupled into a living and organic media through this control system capsule and provides a living media which promotes empathy. We provide user design and feedback results to verify the validity of our hypothesis, and provide not only system results but generalized design frameworks for empathetic living media in general.