Music education for Deaf and hard-of-hearing (DHH) learners remains challenging because access to auditory cues is constrained and instruction is often shaped by hearing-centered pedagogical and communication structures. Prior work has explored haptics for music, yet offers limited situated knowledge of how haptic cues should be designed and negotiated within authentic instruction. We address this gap through a co-design method centered on reconfigurable haptic prototyping. Using modular tactors that can be rapidly repositioned and retuned, we conducted four co-design workshops with educator–learner dyads in South Korea. The workshops involved hands-on trials, bodystorming, and mock teaching enactment to explore music-to-haptic cues within realistic instructional interaction. We show that haptics can support pedagogical fit, make musical concepts tangible as formative in-the-moment support, and function as a shared reference in instruction. We contribute a co-design method, empirical insights into haptics in music instruction, and design implications for DHH music education.

Prototyping with 3D printing depends heavily on virtual modeling, which requires expertise and often leads to scale mismatches and inefficient iteration. Physical deformation through heating is possible but challenging, as heating is difficult to control and deformation remains complex. We introduce TF-Shell, a thermoformable shell that enables repeatable, localized thermoforming of 3D-printed prototypes. Leveraging shape-memory properties, TF-Shell allows 3D-printed objects to achieve volumetric deformation, restoration, and shape memorization within the physical prototyping process. With customizable features, it can be embedded into free-form models through a design tool. A user study shows that TF-Shell provides intuitive, convenient physical modification and expands iteration beyond virtual modeling. Technical evaluations confirm its thermoformability and repeatability, establishing TF-Shell as a practical approach for integrating physical deformation into 3D printing workflows.

Conversational search, powered by Large Language Models (LLMs), has rapidly become a dominant mode of information seeking. While LLMs reduce the effort of information seeking, they also introduce the risk of distorted information deceptively embedded in responses. Prior work has sought technical mitigations, but such distortion cannot be fully eliminated. We therefore shift the focus to the user level, supporting users in deliberately engaging with information when reading LLM responses. We conducted a user study with frequent conversational search users (N=16), comparing a baseline with two probes—LLM-box (LLM-as-a-judge feedback) and Thinking-box (checkpoints from hallucination patterns)—to examine how these probes influenced users’ recognition of distorted information and their experience of guidance. Our findings indicate that even indirect suggestions significantly improved users’ ability to filter distorted information, while also revealing that guidance must be selective to prevent cognitive overload. These insights point to design implications that enable more deliberate user engagement with LLM responses.

Figure 1: An illustration showcasing the typical dialogue patterns between the child and the agent across the three conditions of HelloCuriosity, a system designed to investigate how agents support children's inquiry in exhibits AbstractCompetent responders to children's questions foster high-quality inquiry.While conversational agents (CAs) have been explored in various roles to support questioning, little research has examined how different agent responses shape inquiry.Children engage in exhibits not only with agents but also with surrounding objects, requiring CAs to support both inquiry and behavioral engagement-an aspect that remains underexplored.We conducted a comparative user study (N=36, ages 6-12) examining three agent types modeled after insights from our formative study: a reactive responder, a facilitative responder, and an active peer.Findings

The perception of pitch and engagement in melodic singing significantly contribute to the appreciation and comprehension of music. Notably, for hard-of-hearing individuals, perceiving pitch and singing play crucial roles as effective aids in auditory rehabilitation. However, the limited perception of pitch, stemming from the constraints of hearing devices, poses challenges in the learning process of recognizing and vocalizing melodies. In this study, we developed a wearable vibrotactile glove, TTS+, designed to convert musical tones into the corresponding vibrotactile patterns, complemented by an associated training application. After ten training sessions with our system, the participants exhibited notable improvements in pitch perception and the capacity to reproduce melodies through singing compared with their pre-test baseline without vibrotactile feedback. Beyond quantitative outcomes, findings from user experiences obtained through user feedback during training and post-test interviews offer valuable perspectives and constraints for our study, which may be further investigated in future studies.

The pandemic has contributed to the increased digital content development for remote experiences. Notably, museums have begun creating virtual exhibitions using 360-videos, providing a sense of presence and high level of immersion. However, 360-video content often uses a linear timeline interface that requires viewers to follow the path decided by the video creators. This format limits viewers’ ability to actively engage with and explore the virtual space independently. Therefore, we designed a map-based video interface, Viewer2Explorer, that enables the user to perceive and explore virtual spaces autonomously. We then conducted a study to compare the overall experience between the existing linear timeline and map interfaces. Viewer2Explorer enhanced users’ spatial controllability and enabled active exploration in virtual museum exhibition spaces. Additionally, based on our map interface, we discuss a new type of immersion and assisted autonomy that can be experienced through a 360-video interface and provide design insights for future content.

As interest in materiality has increased in human-computer interaction (HCI), a growing body of research has emerged on the use of smart materials to enhance interactivity. In particular, thermochromic materials, which change color in response to environmental temperature, have drawn attention. However, despite their dynamicity and interactivity, limited studies have dug into their playful potential. This paper presents Thermo-Play, a series of four projects—Picky Tamagotchi, Alphabet Blocks, Shake My Hand, and Gradual Display—that examine the potential of thermochromism in the sense of play and its implications for design and HCI. Evaluating the four projects, we identify the elements that make thermochromic materials playful, such as physicality, color-dynamicity, and gradualness, and the limitations of these materials. Through the findings of Thermo-Play, we expect to discover new opportunities for thermochromic materials in designing playful interactions.

Animal live streaming (i.e., live streams that feature animals) has recently gained popularity. Nevertheless, little is known about the effect of leveraging viewer-animal interaction on the live streaming experience. In this regard, we introduce MeowPlayLive, a live streaming system that allows viewers to interact with the cat in the live stream by sending voice messages. Since messages appear as moving objects on a tablet screen perceivable by the cat, viewers can win a chance to get heard when the cat decides to tap on the objects. By deploying MeowPlayLive in actual live streams, we found that voice message-based viewer-animal interaction motivates viewers to become active participants, renders the live stream more enjoyable, enhances viewers’ attachment to the cat, and positively transforms the streamer-cat relationship. Our results suggest that viewer-animal interaction should be animal-driven, collaborative, and rewarding. We hope to inspire researchers to explore new forms of technology-mediated human-animal interaction.

The 3D pen has become a popular crafting tool where hands-on deformations are largely engaged. However, as malleable states are invisible, users might be burnt, or their fabrication might fail. We designed a thermochromic 3D filament, ChromoFilament, that displays the malleable states in three different colors according to the associated temperatures. From color design workshops, we identified proper stages of malleability and design considerations for color combinations, which are applied to ChromoFilament. Next, we depict a way to fabricate ChromoFilament from customizing thermochromic ink to extruding with the coated pellets. Finally, we illustrate the users’ distinctive behaviors with ChromoFilament to imply the effects of visible malleable states. We believe that our material-perspective approach, design process, and a series of findings could not only inspire supporting creativity through thermoforming but also heat-based processing in 3D printing.

In recent years, storytelling has attracted attention as a possible important application context for CAs. However, it is difficult to follow and recall the story flow by distinguishing their lines since most story content features multiple characters. We introduce Poly, a physically actuatable CA that implements a 2.5D shape display to help identify multiple characters in a story. To evaluate the effect of adding a novel approach of physical actuation into the conventional color expression technique, we conducted a 2×2 within-subject user study with two levels of shape change (shifting shapes by characters or not) and two levels of color change (shifting colors by characters or not) in a studio setting. The quantitative and qualitative analysis results show that shape change has a significant and exclusive effect on both user acceptance of Poly and narrative engagement of the story, although there is no interaction effect according to color change and shape change.

As young children’s screen time has significantly increased in recent years, their healthy use of digital media is a topic of interest. To support children’s healthy screen use, we are developing Romi, a physical screen peripheral device interface. Romi was designed to help children to transition from screen to out-of-screen by reducing negative experiences when ending screen time. To specify the design concept, we conducted the preliminary interview and experience prototyping focusing on screen-time transition. Based on design attributes which are the peripheral presence, the connection between screen and device, and friendly character, we implemented Romi interface. We believe our approach will provide motivation and inspiration for designing a physical interface for supporting children’s self-regulation.

In this paper, we present a metamaterial structure called thermoformable cells, TF-Cells, to enrich thermoforming for post-print modification. So far, thermoforming is limitedly applied for modifying a 3D printed object due to its low thermal conductivity. TF-Cells consists of beam arrays that affluently pass hot air and have high heat transference. Through heating the embedded TF-Cells of the printed object, users can modify not only the deeper area of the object surface but also its form factor. With a series of technical experiments, we investigated TF-Cells’ thermoformability, depending on their structure’s parameters, orientations, and heating conditions. Next, we present a series of compound cells consisting of TF-Cells and solid structure to adjust stiffness or reduce undesirable shape deformation. Adapting the results from the experiments, we built a simple tool for embedding TF-Cells into a 3D model. Using the tool, we implemented examples under contexts of mechanical fitting, ergonomic fitting, and aesthetic tuning.

This study aims to clarify the effect of non-speech sound augmentation (i.e., everyday and instrumental sounds) on outdoor play for children, where has been lacking in empirical examination. In a within-subject observational study, sixteen children (ages 10-11) were divided into four equally sized groups and equipped with SoundWear, which is a wearable bracelet that allowed them to explore sounds, pick a desired sound, generate the sound with a swinging movement, and transfer the sound between multiple devices. Both the quantitative and qualitative results revealed that augmenting everyday sounds led to distinct play types with differences in physical, social, and imaginative behaviors, whereas instrumental sounds were naturally integrated into traditional games. Thus, sound augmentation with specific digital design features (e.g., transparent technology to provide new perspectives, margin for interpretation, and ownership through a sense of achievement) is significant for shaping distinctions in digitally enhanced play and requires considerable design attention.

Many museums have begun to use mobile devices to provide interactive content to visitors. However, such devices can also easily distract visitors, drawing their attention away from the exhibits. In this pictorial, we show the steps in designing Scienscope-a hand-held mediation device that can be used as a stethoscope to explore additional information hidden within exhibits at close range. As Scienscope hovers on authentic objects, it provides digital content and audio-haptic feedback, reinforcing the visitors' exploratory behaviors. After interviewing museum experts and bodystorming the museum space, we followed an iterative design process using low-fi prototyping to explore the users' experience. Three application scenarios were devised to show how Scienscope can be used for different types of exhibits. We discussed how visitors respond to and explore hidden information about exhibits at close range, which can be helpful for museum curators and designers.

We present DexController, which is a hand-held controller leveraging grasp as an additional modality for virtual reality (VR) game. The pressure-sensitive surface of DexController was designed to recognize two different grasp-poses (i.e. precision grip and power grip) and detect grasp-force. Based on the results of two feasibility tests, a VR defense game was designed in which players could attack each enemy using the proper weapon with a proper level of force. A within-subject comparative study is conducted with a button-based controller which has the same physical form of DexController. The results indicated that DexController enhanced the perceived naturalness of the controller and game enjoyment, with having acceptable physical demand. This study clarifies the empirical effect of utilizing grasp-recognition on VR game controller to enhance interactivity. Also, we provide insight for the integration of VR game elements with the grasping modality of a controller.

We developed a hand-held controller named DexController, leveraging grasp as an additional input modality for virtual reality(VR) game. The pressure-sensitive surface of DexController could recognize two different grasp-poses (i.e. precision grip and power grip) and detect grasp-force. For demonstration, we designed a VR defense game in which players should attack different virtual enemies using the proper weapon with a proper level of force. User study confirmed that utilizing meaningful information of grasping facilitates natural mapping with game contents, which led VR game users to experience enhanced presence and enjoyment.

Interpersonal touch, one of the most primitive social languages, is an excellent design element frequently used in interaction design. In this paper, we present a richer understanding of it by using spatial factors and social relations among people, which has rarely been explored in interactive systems. We designed an interactive installation called "TouchBranch" where players can move light between branches placed at various distances by connecting their bodies. The user studies were conducted with 21 groups consisting of intimates, acquaintances, and strangers. We observed a change in the interpersonal touch pattern and touch tolerance according to each factor. Interestingly, the effect of the social relation was dramatic, but that of the spatial factor was not quantitatively significant. Nevertheless, we discovered that spatial factor can influence the interpersonal touch experience. Based on the results, we discuss in this paper the influence of two factors on the interpersonal touch that stands out in the context of interactive systems.

Despite the popularity of fish as pets, there is little knowledge available about the fishkeeping experience and the related interactions. In this regard, this study aims to look into the experience of fishkeeping by supporting people's actions through a tech-mediated system. Based on the results, an interactive system called BubbleTalk was developed to help people to convey their actions using bubbles into a fish tank. A user study was conducted with BubbleTalk, and the results showed that the interaction through BubbleTalk varied people's behavior, prolonged their interaction and thus reshaped their relationship with fish. Beyond the implications for fishkeeping, we believe that our findings could serve as insight and further motivation for overcoming interactions limited by this physically disconnected environment.

Haptic devices allow a more immersive experience with Virtual and Augmented Reality. However, for a wider range of usage they need to be miniaturized while maintaining the quality of haptic feedback. In this study, we used two kinds of human sensory illusion of vibration. The first illusion involves creating a virtual force (pulling sensation) using asymmetric vibration, and the second involves imparting compliances of complex stress-strain curves (i.e. force-displacement curves of mechanical keyboards) to a rigid object by changing the frequency and amplitude of vibration. Using these two illusions, we developed a wearable tactile device named HapCube, consisting of three orthogonal voicecoil actuators. Four measurement tests and four user tests confirmed that 1) a combination of two orthogonal asymmetric vibrations could provide a 2D virtual force in any tangential directions on a finger pad, and 2) a single voicecoil actuator produced pseudo-force feedback of the complex compliance curves in the normal direction.

Nowadays, it is easy to find concepts for connecting mobile devices and looking at photos together. Despite the increasing interest in multi-device single display groupware (multi-device SDG), most of the existing research is limited to using a rectangular form array to enlarge the display. In this paper, we suggest a new way of assembling mobile devices to create unique forms, such as ring, bar, rectangle and radial shape displays, and investigate their inter-device interaction characteristics. We conducted generative workshops on a game domain to understand the characteristics of those interfaces. Directionality, inter-device space, and tangible interactions were extracted. By applying the derived characteristics, we have developed three different collaborative games using different types of interfaces. Through user studies of the developed games, we refined the characteristics and found additional design issues. The inter-device interaction characteristics and issues of multi-device SDG obtained from this study could be generally applied to collocated multi-mobile interactions.

Knowing your own body movement is an essential element of sports. Recently, the popularization of smartphones has enabled people to easily record their performance in most situations. However, these observations have limited applicability in assisting with a clear understanding of body movement. In this paper, we propose the Motion Log Skateboard, which records and shares non-visible information about body movement that is difficult to obtain through current observation methods in skateboarding. A pressure-sensor matrix on a skateboard deck is used to record the pressure distribution data, which are then played using the video function of a smartphone camera. With this logged data, a user can access the feet positions, pressure intensity, and timing of the foot movements. To verify the proposed concept and determine the specific context of its use, an experimental session and interviews were conducted with skateboarders of various skill levels. Based on the results of this research, the shared experiences of non-visible information, which is perceived differently depending on the individual, are expected to become a standard for exploring and training body movement.

This paper introduces an interactive snowboard concept, Motion Echo Snowboard, which provides visually augmented feedback for body movements during snowboarding. The proposed system augments the user experience with a real-time visual display of performer weight distribution on the upper side of the snowboard. We conducted two user studies: an individual usage and a lesson situation. Based on the results, we confirmed that the Motion Echo Snowboard concept could enhance movement perception and motivation as being the starting point for exploring body movement in both situations. Moreover, it could generate new behavior patterns and could establish student-instructor trust as a medium of movement during lessons.

A virtual touch screen concept using an optical see-through head-mounted display has been suggested. With a virtual touch screen, the user's direct-touch interactions are allowed in much the same way as a conventional touch screen, but the absence of haptic feedback and physical constraint leads to poor user performance. To overcome this issue, we developed a wearable haptic device, called HapThimble. It provides various types of haptic feedback (tactile, pseudo-force, and vibrotactile) to the user's fingertip and mimics physical buttons based on force-penetration depth curves. We conducted three experiments with HapThimble. The first experiment confirmed that HapThimble could increase a users' performance when conducting clicking and dragging tasks. The second experiment revealed that users could differentiate between six types of haptic feedback, rendered based on different force-penetration depth curves obtained using HapThimble. Last, we conducted a test to investigate the similarity between the physical buttons and the mimicked haptic buttons and obtained a 90.3% success rate.

The two main tasks of a smart TV GUI are menu navigation and free pointing. Traditional remotes with directional keys are suitable for menu navigation but may not be so for free pointing. More recent remotes with a two-dimensional (2D) pointing device are suitable for free pointing but may not be so for menu navigation. To support both types of tasks well, we devised a new input device called TouchRoller. We expect that it can support both types of tasks well because it has a separable control structure and a continuous input property. A comparative user study showed that the performance of TouchRoller is comparable to that of directional keys for menu navigation and 2D pointing devices for free pointing. In addition, it was most favored by the participants, and NASA TLX test results showed that TouchRoller demands the lowest task load.

Computing technologies are increasingly designed to support motor-impaired people with physical rehabilitation. Although it is important to reflect patients' motivation to maintain effectiveness in therapy, these studies mostly show that these technologies can increase rehabilitation effectiveness by providing patients with certain stimulations repetitively. Patients can lose interest because of passive motion from limited stimulation.

A light switch and its spatial mapping constitute an unsolved, user interface problem introduced to the realm of Human Computer Interaction by Norman. In this paper, we introduce the Previewable Switch which is a light switch enhanced with a touch sensor with an ability to provide a user a glimpse of which lights will turn on/off prior to compressing the switch. Such concept of communicating a result prior to taking an action is known as feedforward [6]. Feedforward is an important element to be considered in the user interface design as it provides clear and instant affordance of what will occur before the next action. We present the findings from the study in regard to behavior and spatial mapping. We will discuss the effect of feedforward as a design element in a user interface.

This paper presents a set of cubes for modular design that act as a meaningful trial for the democratization of design and manufacturing. Called CUBEMENT, the cubes are compatible with existing building platforms such as LEGO Technics, and physical computing components. CUBEMENT consists of a power supply and components for mechanism building that can support other cubes or extend to physical computing. CUBEMENT components can be connected and interlocked with one another by simply snapping together small magnets. The aim of CUBEMENT is to create a seed movement, expand the application of structural mechanisms further, and control that movement by embracing other platforms within its platform. Therefore, it will be able to simultaneously support the sketching of movement from mechanism building and physical computing. CUBEMENT also plays a role as a tangible communication medium for intangible mechanism movement.

A physical keyboard key has three states, whereas a touch screen usually has only two. Due to this difference, the state corresponding to the touched state of a physical key is missing in a touch screen keyboard. This touched state is an important factor in the usability of a keyboard. In order to recover the role of a touched state in a touch screen, we propose the TapBoard, a touch screen software keyboard that regards tapping actions as keystrokes and other touches as the touched state. In a series of user studies, we validate the effectiveness of the TapBoard concept. First, we show that tapping to type is in fact compatible with the existing typing skill of most touch screen keyboard users. Second, users quickly adapt to the TapBoard and learn to rest their fingers in the touched state. Finally, we confirm by a controlled experiment that there is no difference in text-entry performance between the TapBoard and a traditional touch screen software keyboard. In addition to these experimental results, we demonstrate a few new interaction techniques that will be made possible by the TapBoard.

We introduce a new type of organic user interface that displays a 3D robotic creature, ‘MoleBot’ to provide a ludic experience inspired by traditional board games. To ensure fluid motions of the molehills cast by the ‘MoleBot’, the table surface combines horizontal rigidity with the vertical flexibility of over 15,000 movable pins. Users are enabled to kinetically interact with this creature via a joystick or gestural commands. We conducted user study sessions with 12 participants and classified the observed spontaneous play activities into 4 distinct categories: (1) enjoying simple ludic experience, (2) competing in skills, (3) mimicking realworld sports and (4) playing with a companion. In addition, a focusgroup interview with six video scenarios was conducted to explore the idea of potential applications and it was suggested that the ‘MoleBot’ can be used in interactive board-gaming environments and kinetically informative tabletops.

We present an air-contained display medium that can be directly deformed and spatially moved by various physical interaction techniques for interactive games. We first investigated familiar objects in our everyday lives that allow users to easily anticipate the idea of exertion interaction. We then introduce a novel concept of interactive medium, dubbed Inflated Roly-Poly, which consists of an inflated body with a roly-poly structure. This device receives physical input, provides passive haptic feedback and allows spatial interaction. We discuss a number of interaction techniques with game applications on Inflated Roly-Poly that presents an engaging experience through full-body interaction. Finally, we conducted an experience workshop with four participants. The workshop proved that an inflated screen coupled with a roly-poly structure exceeds the capabilities of the rigid touch screens in terms of engagement in physical interaction.

We explored the possibility of touch-screen-like interaction with a remote control in the TV-viewing environment. A shadow representing the user's thumb touches the screen, presses a button, flicks a cover-flow list, and draws a simple stroke, while the thumb stays and moves on and above the touchpad. In order to implement the concept we developed an optical touchpad for tracking the thumb hovering over its surface, and designed a TV application to demonstrate possible new interaction styles. Throughout two iterations of prototyping, we corrected some of our false expectations, and also verified its potential as a viable option for a TV remote control. This paper presents technical issues and requirements for the hover-tracking touchpad and a complete report of our user studies to explore touch-screen-like interaction for the TV.

We propose and demonstrate Kinetic Tiles, modular construction units for kinetic animations. Three different design methods are explored and evaluated for kinetic animation with the Kinetic Tiles using preset movements, design via animation toolkit, and design via direct input. It is expected that the Kinetic Tiles, as a new design and architecture material, will assist designers to introduce kinetic expressions to the surfaces of everyday objects and spaces.

Inflatable Mouse is a volume-adjustable user interface. It can be inflated up to the volume of a familiar mouse, but be deflated and stored flat in a PC card slot of a laptop computer when not in use. Inflatable Mouse functions just like a typical mouse; moreover, it provides new interaction techniques by sensing the air pressure in the balloon of the mouse. It also addresses some issues associated with pressure-sensing interactions such as the lack of bi-directional input and the lack of effective feedback. Moreover, it can be used as both a control tool and a display tool. In this paper, the design of an Inflatable Mouse prototype is described and potential application scenarios such as zooming in/out and fast scrolling using pressure control are explained. We also discuss the potential use of Inflatable Mouse as an emotional communication tool.

Computer aided design applications have become designers' inevitable tools for expressing and simulating innovative ideas and concepts. However, replacing traditional materials and mock-ups with 3D CAD systems, designers are faced with the intangibility problem, unable to physically interact with test products in early stages of design process. As a touchable and graspable interface based on 3D CAD data, we propose augmented foam, which applies augmented reality technologies to physical blue foams. Using augmented foam, a blue foam mock-up is overlaid with a 3D virtual object, which is rendered with the same CAD model used for mock-up production. We presented a method to correct occlusions of the virtual products by user's hand. Augmented foam was tested for a mug design and a cleaning robot design. Designers were able to inspect and evaluate the design alternatives interactively and efficiently.