Sound offers unique affordances for interaction designers, enabling immediate, embodied, and expressive experiences. Yet sound is often treated as peripheral, limited to simple notifications rather than being explored as a central design material. Designing meaningful auditory feedback typically requires low-level programming, audio-specific expertise, and fragmented toolchains that hinder early-stage experimentation. We present TESI (Tangible and Embodied Sound Interaction), a modular toolkit that reduces these barriers by combining sensor-integrated hardware, open-source middleware, and a visual programming environment for real-time sensor-to-sound mapping. TESI enables interaction designers to prototype adaptive soundscapes across networked tangible interfaces with minimal setup. We describe the toolkit’s architecture, illustrate its use through examples, and discuss future directions for embodied sound design in distributed systems.

This studio invites participants to explore tangible and embodied interaction beyond standalone interfaces into interconnected ecologies of smart objects, focusing on sound as a primary design element. Through a series of hands-on experimentations, participants will prototype with a modular toolkit, exploring how sound can move beyond simple notification functions to create rich and adaptive soundscapes. Drawing from the practices of digital lutherie and sound design, in particular mapping and interaction design strategies, participants will learn how to map sensor data to sound using interactive machine learning techniques based on artificial neural networks. This studio explores sustainable practices by adopting a modular approach to create adaptable systems, while promoting community engagement, open source contributions, and wider accessibility.

As the datafication of our personal lives increases, researchers have started to critique what we consider as data. Where data are often seen from a reductionist perspective –as neutral numbers and graphs– theory suggests that data are messy, subjective, and pertaining to more than the human alone. They are experienced and lived with, something which is often not accounted for in personal data. To explore what people themselves consider data, sixteen interlocutors participated in a cultural probe study where they visually documented what they considered data in their everyday lives. Our analysis indicates that data can be disciplinary, social, and extend beyond ourselves, incorporating more-than-human aspects. Data are often regarded for entertainment, work, and wellbeing purposes. As interlocutors actively engaged and lived with data, we end the pictorial with a working definition of personal data, which acknowledges people's active roles in their data creation.

External Human-Machine Interfaces (eHMIs) have been evaluated to facilitate interactions between Automated Vehicles (AVs) and pedestrians. Most eHMIs are, however, visual/ light-based solutions, and multi-modal eHMIs have received little attention to date. We ran an experimental video study (N = 29) to systematically understand the effect on pedestrian’s willingness to cross the road and user preferences of a light-based eHMI (light bar on the bumper) and two sound-based eHMIs (bell sound and droning sound), and combinations thereof. We found no objective change in pedestrians’ willingness to cross the road based on the nature of eHMI, although people expressed different subjective preferences for the different ways an eHMI may communicate, and sometimes even strong dislike for multi-modal eHMIs. This shows that the modality of the evaluated eHMI concepts had relatively little impact on their effectiveness. Consequently, this lays an important groundwork for accessibility considerations of future eHMIs, and points towards the insight that provisions can be made for taking user preferences into account without compromising effectiveness.

This paper explores (1) the role of metaphors in physical data representations and (2) the concept of tacit data: implicitly known data which are hard to uncover. In a semester course with twenty-three students, five teams explored how to represent self-chosen ‘tacit data’ in a visualisation, haptification, and dynamic physicalisation. Throughout these phases, our notion of tacit data evolved, resulting in a proposed working definition. Moreover, we noticed that metaphors played an increasingly important role. Based on analysis of students’ work and interviews with them, we found that tacit data and physical data representations need metaphors. For haptifications and physicalisations, metaphors help to circumvent limitations, curate data, and communicate to the audience. As tacit data were seen as ‘soft’ and difficult to quantify, metaphors made the data workable. Furthermore, tacit data benefit from physical representations, which offer further dimensions to represent the feeling and intimate aspects of data.

In design, we often deal with complex problems that require participatory, embodied sensemaking to solve. There are abundant design tools available – both physical and digital – to support these activities. While digital tools have continued to gain presence in the design processes over the last decades, due to, e.g., widened availability, improved flexibility, and the potential to increase productivity, physical tools and analogue practices still hold a solid place in the design process for many designers. To bridge the gap between the physical and digital, and to benefit from the best of both worlds in tools for design, hybrid tools are being developed. This paper aims to identify design opportunities for future hybrid design tools, by exploring the characteristics of designing for participatory, embodied sensemaking, in the context of physical, digital, and hybrid tools.

Positive peer interaction in nursing homes has been consistently recognized as essential to residents’ life quality. However, low rates of resident-to-resident interaction were found to be pervasive. Our research explores the potential of applying public display systems to promote residents’ unplanned co-located interaction. This article describes the design and assessment of “Reading-to-Sharing” (R2S): a tabletop display system intended to improve nursing home residents’ social interaction by enhancing their public reading experience. R2S was assessed via supervised field trials, in which the participants were invited to experience R2S in real-life settings with necessary assistance. The objectives were mainly to investigate the participants’ engagement with R2S, user experience and the potential impact on residents’ social behaviors and feelings. The result showed that R2S was capable of engaging the participants in content viewing and sharing. It was effective in catalyzing and facilitating their social interaction. The participants’ perceived user experience was primarily favorable. Although R2S was anticipated to increase the participants’ mutual closeness, no statistically significant change was seen. The key implications were highlighted to guide the design of public display systems in this context.

Abstract Mementoes act as emotional companions that anchor stories. Older adults typically have a rich knowledge of family mementoes. However, storytelling and preservation of mementoes are still problematic for them: their mementoes are still mostly in physical format, which is difficult to share and preserve. Additionally, digital applications and websites for sharing mementoes usually are inaccessible for them. As a result, they spend much time collecting mementoes, but spend less time on telling and recording the related stories. In response to this, we report our study driven by the research questions: Rq1: What are the characteristics of older adults’ intergenerational memento storytelling? And Rq2: In which ways could a tangible display facilitate intergenerational memento storytelling for older adults? We designed a tangible device named Slots-Memento. We first conducted a preliminary evaluation to refine the prototype. In the field study, eight pairs of participants (each pair consisting of an older adult and his/her child) were recruited to use the prototype for around 1 week. Semi-structured interviews were then conducted both with the older adults and their children. Subsequently, mementoes collected were categorized and analyzed. Stories collected were firstly transcribed, then were conducted with structural and interactional analysis. In the concluding discussion, we present abstract implications for the research questions: two tables summarizing characteristics of their intergenerational memento storytelling, and related strategies of designing a tangible display individually.

Audio-frequency induction loops (AFILs) are commonly used as an assistive listening technology for hard-of-hearing individuals. They generate an electromagnetic field proportional to a sound source receivable by hearing aids. Our interactive system, the Sound of Space, is based on AFILs that generate a multi-dimensional soundscape in space. Cochlear implant (CI) listeners and hearing-aids wearers can experience the soundscape through bodily movement, whereas hearing individuals can experience it via a corresponding tangible device. While typical AFIL installations transmit a single sound source, in our interactive system we implement overlapping loops and their interference to locate multiple synchronised audio sources (i.e., corresponding electromagnetic fields) in space. The designed system is installed permanently in an integrative school for deaf, hard-of-hearing and hearing students and teachers. In this pictorial, we illustrate our design and implementation process and contribute our learnings of using AFILs as a design material for architectural interactivity.

Passwords have become a ubiquitous part of our everyday lives, needed for every web-service and system. However, it is challenging to create safe and diverse alphanumeric passwords, and to recall them, imposing a cognitive burden on the user. Through consecutive experiments, we explored the movement space, affordances and interaction, and memorability of a tangible, handheld, embodied password. In this context, we found that: (1) a movement space of 200 mm × 200 mm is preferred; (2) each context has a perceived level of safety, which—together with the affordances and link to familiarity—influences how the password is performed. Furthermore, the artefact’s dimensions should be balanced within the design itself, with the user, and the context, but there is a trade-off between the perceived safety and ergonomics; and (3) the designed embodied passwords can be recalled for at least a week, with participants creating unique passwords which were reproduced consistently.

Institutional care settings are often described as places where residents suffer from social isolation. In the past decades, conventional social interventions changed very little and had many limitations. Our research explores the potential of integrating technologies into public caring environments. In this paper, we present "R2S", a flexible platform in care homes to support residents' social interaction through augmenting public print media products. It aims to provide caregivers a convenient way to transform any print media into interactive surfaces, which can enhance residents' public reading experience and trigger communications. This paper describes the design motivation, the design and prototype implementation, preliminary user feedback and future plans.

Over the past decades monetary transactions have become increasingly dematerialized. Nowadays payments should be "frictionless", as easy as possible, for example through NFC or biometric payment methods. Although frictionless payment methods hold advantages in terms of safety, convenience and privacy, they also pose problems as the inherent physicality of cash is shown to help people with budgeting and expense tracking. In this paper we describe our ongoing work in creating more mindful interactions, in this case by re-physicalizing part of digital transactions with the aim of combining the advantages of digital payment methods with those of physical money. We present the evaluation of a haptic interface for in-store digital payments. Results from a four-week lab evaluation indicate that people are capable of getting a feel for the absolute values of a payment as presented through our interface.

Mementos carry personal symbolic meaning and can be used to privately reflect on the past or share memories[28]. Older adults spend much time collecting mementos but spend less time telling stories behind the mementos. Once they pass away, stories behind mementos vanish with their owners. In this paper, we present Slots-Memento, a tangible device aiming to facilitate intergenerational story sharing and preservation for older adults. It builds on the metaphor of slot-machine, and integrates functions of memento photo display, story recording, and preservation. Our design process started with context inquiry, older adults and young adults were recruited, aiming to understand the status quo of their memento storytelling, and define design requirements. A preliminary evaluation was conducted, discussion and future work are in the final part.

In this paper we present DOK, a tangible conversation tool aimed at empowering child patients during medical triads. Current medical triads tend to exclude child patients, making them bystanders of their own illness: triads are highly wordy, cognition-heavy and mostly take place between doctor and parent. In this ongoing research we aim to develop a tool that empowers child patients by allowing them more input and initiative in the conversation. We present our theoretical backbone, point out opportunities for tangibles, and describe the current version of DOK.

Prototypes and prototyping have had a long and important history in the HCI community and have played a highly significant role in creating technology that is easier and more fulfilling to use. Yet, as focus in HCI is expanding to investigate complex matters of human relationships with technology over time in the intimate and contested contexts of everyday life, the notion of a 'prototype' may not be fully sufficient to support these kinds of inquiries. We propose the research product as an extension and evolution of the research prototype to support generative inquiries in this emerging research area. We articulate four interrelated qualities of research products inquiry-driven, finish, fit, and independent and draw on these qualities to describe and analyze five different yet related design research cases we have collectively conducted over the past six years. We conclude with a discussion of challenges and opportunities for crafting research products and the implications they suggest for future design-oriented HCI research.

In this paper we describe the development and evaluation of three kitchen blenders that were specifically designed to stimulate mindfulness in interaction, that is: engagement with, and care for what you are doing. We find that the directness we used to have preparing our food has been sacrificed to efficiency and ease of use, which does not match our current zest for 'slow food' and 'slow cooking'. We argue that most of our kitchen appliances make us less engaged in the act of and less caring for cooking. In order to counter this we see opportunities for a more tangible or embodied interaction style where expressive input leads to expressive output. In order to research this argument we have developed three embodied kitchen blender interaction styles and compared these to a more traditional blender interaction. Preliminary findings suggest that more embodied interaction styles do indeed lead to more mindful engagement in interaction.

In this paper we discuss our recent research into notation systems for interaction design. Inspired by music notation - a standardized yet open system with which a composition (i.e., a musical design) can be communicated - we suspect that similar notation systems can be designed for interaction design. We describe our research approach to this suspicion and describe several outcomes of our research, which all lean heavily on tangibility, manipulability and shareability.

This paper presents a work in progress design case that is used to exemplify how a phenomenology-inspired perspective on time can impact the design of highly interactive systems and products. The design presents a calendar with a re-orientated layout that is based on a bodily relationship with time, rather than on the more established linear chronological layout found in most products and systems. The presented design offers a promising perspective as well as emergent technical issues and questions. With this work we hope to inspire interaction designers to consider embodied mechanisms.

In this paper, we describe our ongoing work towards understanding how interactive comics could affect time perception. An interactive comic story, The Dreaming Wine, has been created based on our current understanding of interactive comics, time perception and the relation between both. As a starting point for our research, we conducted an experiment on whether the amount of panels would affect the time perception of the comic reader. A list of principles was applied in splitting comic panels. The findings of the experiment are expected to provide a better understanding of how panels in interactive comics have an effect on the reader’s time perception.

In this paper we present the final research prototype of LinguaBytes, a tangible interface aimed at stimulating the language development of non- or hardly speaking children between 1 and 4 years old. LinguaBytes was developed in a three-year Research through Design process in which five incremental prototypes were designed, built and evaluated in real-life settings. In this paper we present the original starting points of the project, describe our method and illustrate the resulting end-design using example scenarios of use. We give an overview of the most significant findings at the ten-month evaluation moment, after which we reflect on the original starting points and assess whether they hold up.

Here we present LinguaBytes, a modular, tangible play-and-learning system developed to stimulate the language and communication skills of toddlers (with a developmental age between 1 and 4 years) with multiple disabilities. The typical characteristics of these children and their common learning environment call for intelligent systems that are quickly and easily adaptable, or, even better, adjust themselves proactively. Over the last two and a half years we have designed and tested such a system within LinguaBytes. In this paper we will outline starting points, indicate the opportunities of Tangible Interaction and finally give an overview of the current LinguaBytes prototype.

The LinguaBytes-project is a three-year research project aimed at developing an adaptive, interactive, tangible play and learning system to stimulate the early language competencies of toddlers at a developmental age between 1 and 4 years - with severe motor and/or multiple disabilities. It serves as the main case study for the first author's PhD research on designing adaptive products. We believe that children from the abovementioned, highly heterogeneous user group could benefit greatly from products or interfaces that could either be adjusted (adaptability) or adjust themselves (adaptivity) to their individual needs. Advances in technology are gradually enabling designers to create such products, but guidelines that help designers do this are scarce. Designers need more insight in the implications of adaptivity and adaptability on the form and content of their design in order to make good design decisions. Using a research-through- design method, we try to generate some of the knowledge that can help designers, not only when designing products or interfaces for this specific user group, but also for heterogeneous user groups in general.

In this paper we focus on two aspects of Tangible Interaction that have our particular interest: 1) the added value of tangibility when designing interfaces for toddlers and 2) the value of actuators. Especially the latter is something that in our opinion has been under-investigated within the field of Tangible and Embedded Interaction. In this paper we will address the abovementioned topics by giving examples from the LinguaBytes project, which is aimed at developing an intelligent interactive play and learning environment for toddlers with multiple disabilities. These two aspects of Tangible Interaction have our particular interest since we see that multi-handicapped children could benefit highly from Tangible Interaction, but often lack the necessary bodily skills. Using actuators could offer these children possibilities to become more autonomous, thus enhancing their self-esteem and motivation. We feel that our work could not only benefit multi-handicapped toddlers in particular, but could also be used to design interactions that are more respectful to heterogeneous users in general.

Interactive products can help very young multi-handicapped children (1-4 years) develop their language and communication skills, under the condition that they are optimally tuned to the individual child. This has great consequences for design, since this kind of interactive products need to be adaptive to the child's development, possibilities, interests and needs. There are currently hardly any guidelines for designing adaptive interactive tangible products for such a heterogeneous user group. Through LinguaBytes, a three-year research project aimed at the development of an adaptive interactive toy for stimulating language and communication skills of multi-handicapped toddlers, we want to establish a theoretical framework, including guidelines and tools, for designing complex interactive products.