We introduce Painting Inferno, a novel method for controlling heat and stiffness using highly electrically conductive carbon nanomaterial heating paint. Heat has found widespread applications in thermochromic displays, shape-changing interfaces, haptic devices, and materials with adjustable stiffness. Although Joule heaters based on heating circuits using electrically conductive materials have been widely used, the complex design and fabrication processes limit the freedom to create custom heaters in scale, shape, and material. As an alternative Joule heating method, we explore the potential of carbon nanomaterial heating paints, which enable the rapid generation of uniform heat at low voltages. We present simple fabrication methods for creating handmade heaters using off-the-shelf materials and cutting machines and demonstrate the feasibility of crafting heaters with complex shapes and grid-array configurations. Leveraging the heating paint’s compatibility with various materials, we showcase the versatile applications for interactive thermal displays, stiffness modulation devices, and pneumatic interfaces for stiffness-shape transformations.

During recruitment interviews, the facial impressions of the interviewers likely affect the nervousness of the interviewees and make it difficult to conduct fair and consistent interview processes. To minimize the difference in facial impressions among interviewers, we have investigated a method to convert an interviewer’s face into an avatar. In this study, we used a digital full-face mask display capable of replacing the wearer’s face with an avatar in the real world. By reproducing the interviewer’s facial expression with an avatar in real time, we investigated the effect of avatar appearance on interviewee’s nervousness during interviews. Two types of avatars (a dignified face and a gentle face) were applied to three male interviewers in different age groups (10s, 20s and 60s). We compared the level of interviewee’s nervousness between before and after augmenting interviewer’s face with avatar. 172 college students were recruited as interviewees to assess the variation in the level of nervousness. Our experimental results show that avatar appearance can elicit more unique and consistent impressions than the interviewer’s real face and reduce the variation in interviewee’s nervousness level across interviewers.

We present Flowcuits, a DIY fabrication method to prototype tangible, interactive and functional electrical components by manipulating liquid metals. The prototypes afford both physical and visual interactions to demonstrate the inner working mechanics of fundamental electronic elements, which enables tangible and playful learning. The fabrication process follows simple imprinting and sealing of fluidic circuits with a 3D-printed stamp on an accessible moldable-substrates such as ‘Blu Tack’. Utilizing conductive gallium indium liquid metal, we demonstrated interactive and re-configurable electronic components such as switches, variable resistors, variable capacitors, logic gates and pressure sensors. In this paper, we present the design analogy of Flowcuits, DIY fabrication approach including a parametric 3D stamp design toolkit and results from a technical evaluation. The stamps are printed with a low-cost 3D printer and all the materials are inexpensive and reusable, enabling Flowcuits to be easily used without any advanced lab facilities.

This paper presents a thin digital full-face mask display that can reflect an entire facial expression of a user onto an avatar to support augmented face-to-face communication in real environments. Although camera-based facial expression recognition technology has enabled people to augment their faces with avatars, application was limited to face-to-face communication in virtual environments. To enable digital facial augmentation with an avatar in a real space, we propose a digital face mask display system that integrates a lightweight flexible display with a thin facial expression recognition system. The thin wearable facial expression recognition system was implemented with photo reflective sensor arrays which can measure facial expressions at 40 feature points distributed across an entire face. We investigated a ten-class facial expression identification model based on an SVM training algorithm. The trained model achieved an average accuracy of 79% when identifying the facial expressions of multiple users. User experiments indicated that the proposed thin digital full-face mask display allows the wearer to control the facial expression of the avatar with a fast response rate and create a positive sense of self-agency and self-ownership toward the augmented avatar face.

Touchscreens are the predominant medium for interactions with digital services; however, their current fixed form factor narrows the scope for rich physical interactions by limiting interaction possibilities to a single, planar surface. In this paper we introduce the concept of PickCells, a fully re-configurable device concept composed of cells, that breaks the mould of rigid screens and explores a modular system that affords rich sets of tangible interactions and novel across-device relationships. Through a series of co-design activities -- involving HCI experts and potential end-users of such systems -- we synthesised a design space aimed at inspiring future research, giving researchers and designers a framework in which to explore modular screen interactions. The design space we propose unifies existing works on modular touch surfaces under a general framework and broadens horizons by opening up unexplored spaces providing new interaction possibilities. In this paper, we present the PickCells concept, a design space of modular touch surfaces, and propose a toolkit for quick scenario prototyping.

We present Tangible Drops, a visio-tactile display that for the first time provides physical visualization and tactile feedback using a planar liquid interface. It presents digital information interactively by tracing dynamic patterns on horizontal flat surfaces using liquid metal drops on a programmable electrode array. It provides tactile feedback with directional information in the 2D vector plane using linear locomotion and/or vibration of the liquid metal drops. We demonstrate move, oscillate, merge, split and dispense-from-reservoir functions of the liquid metal drops by consuming low power (450 mW per electrode) and low voltage (8--15 V). We report on results of our empirical study with 12 participants on tactile feedback using 8 mm diameter drops, which indicate that Tangible Drops can convey tactile sensations such as changing speed, varying direction and controlled oscillation with no visual feedback. We present the design space and demonstrate the applications of Tangible Drops, and conclude by suggesting potential future applications for the technique.

Ultrasound manipulation is growing in popularity in the HCI community with applications in haptics, on-body interaction, and levitation-based displays. Most of these applications share two key limitations: a) the complexity of the sound fields that can be produced is limited by the physical size of the transducers, and b) no obstacles can be present between the transducers and the control point. We present SoundBender, a hybrid system that overcomes these limitations by combining the versatility of phased arrays of Transducers (PATs) with the precision of acoustic metamaterials. In this paper, we explain our approach to design and implement such hybrid modulators (i.e. to create complex sound fields) and methods to manipulate the field dynamically (i.e. stretch, steer). We demonstrate our concept using self-bending beams enabling both levitation and tactile feedback around an obstacle and present example applications enabled by SoundBender.

We present MistForm, a shape changing fog display that can support one or two users interacting with either 2D or 3D content. Mistform combines affordances from both shape changing interfaces and mid-air displays. For example, a concave display can maintain content in comfortable reach for a single user, while a convex shape can support several users engaged on individual tasks. MistForm also enables unique interaction possibilities by exploiting the synergies between shape changing interfaces and mid-air fog displays. For instance, moving the screen will affect the brightness and blurriness of the screen at specific locations around the display, creating spaces with similar (collaboration) or different visibility (personalized content). We describe the design of MistForm and analyse its inherent challenges, such as image distortion and uneven brightness on dynamic curved surfaces. We provide a machine learning approach to characterize the shape of the screen and a rendering algorithm to remove aberrations. We finally explore novel interactive possibilities and reflect on their potential and limitations.

In this paper, we demonstrate a method for the dynamic 2D transformation of liquid matter and present unique organic animations based on spatio-temporally controlled electric fields. In particular, we deploy a droplet of liquid metal (Gallium indium eutectic alloy) in a 7x7 electrode array prototype system, featuring an integrated image tracking system and a simple GUI. Exploiting the strong dependance of EGaIn's surface tension on external electric voltages, we control multiple electrodes dynamically to manipulate the liquid metal into a fine-grained desired shape. Taking advantage of the high conductivity of liquid metals, we introduce a shape changing, reconfigurable smart circuit as an example of unique applications. We discuss system constraints and the overarching challenge of controlling liquid metals in the presence of phenomena such as splitting, self-electrode interference and finger instabilities. Finally, we reflect on the broader vision of this project and discuss our work in the context of the wider scope of programmable materials.

We present a programmable liquid matter which can dynamically transform its 2D shape into a variety of forms and present unique organic animations based on spatio-temporally controlled electric fields. We deployed a EGaIn (Gallium indium eutectic alloy) liquid metal as our smart liquid material since it features a superior electric conductivity in spite of a liquid state and presents a high dynamic range of surface tension and 2D area controlled by applied voltage strength and polarity. Our proposed liquid metal shape and motion control algorithms with dynamically patterned electric fields realize path tracing organic animation. We demonstrate an interactive 7x7 electrode array control system with a computer vision based GUI system to enable novice users to physically draw alphabet letters and 2D shapes by unique animatronics of liquid metals.