Eating is one of the most multisensory experiences in everyday life. All of our five senses (i.e. taste, smell, vision, hearing and touch) are involved, even if we are not aware of it. However, while multisensory integration has been well studied in psychology, there is not a single platform for testing systematically the effects of different stimuli. This lack of platform results in unresolved design challenges for the design of taste-based immersive experiences. Here, we present LeviSense: the first system designed for multisensory integration in gustatory experiences based on levitated food. Our system enables the systematic exploration of different sensory effects on eating experiences. It also opens up new opportunities for other professionals (e.g., molecular gastronomy chefs) looking for innovative taste-delivery platforms. We describe the design process behind LeviSense and conduct two experiments to test a subset of the crossmodal combinations (i.e., taste and vision, taste and smell). Our results show how different lighting and smell conditions affect the perceived taste intensity, pleasantness, and satisfaction. We discuss how LeviSense creates a new technical, creative, and expressive possibilities in a series of emerging design spaces within Human-Food Interaction.

Centuries of development in optics have given us passive devices (i.e. lenses, mirrors and filters) to enrich audience immersivity with light effects, but there is nothing similar for sound. Beam-forming in concert halls and outdoor gigs still requires a large number of speakers, while headphones are still the state-of-the-art for personalized audio immersivity in VR. In this work, we show how 3D printed acoustic meta-surfaces, assembled into the equivalent of optical systems, may offer a different solution. We demonstrate how to build them and how to use simple design tools, like the thin-lens equation, also for sound. We present some key acoustic devices, like a "collimator", to transform a standard computer speaker into an acoustic "spotlight"; and a "magnifying glass", to create sound sources coming from distinct locations than the speaker itself. Finally, we demonstrate an acoustic varifocal lens, discussing applications equivalent to auto-focus cameras and VR headsets and the limitations of the technology.

Permeable, mid-air displays, such as those using fog or water mist are limited by our ability to shape and control the aerosol and deal with two major issues: (1) the size and complexity of the system, and (2) the creation of laminar flow, to retain display quality. Here we present SonicSpray, a technique using ultrasonic Bessel beams to create reconfigurable mid-air displays. We build a prototype from low-cost, off-the-shelf parts. We explore the potential and limitations of SonicSpray to create and redirect laminar flows of fog. We demonstrate a working prototype that precisely controls laminar aerosols through only 6x6 ultrasound transducers array. We describe the implementation steps to build the device, verify the control and projection algorithm for the display, and evaluate its performance. We finally report our exploration of several useful applications, in learning, entertainment and arts.

Ultrasound manipulation is growing in popularity in the HCI community with applications in haptics, on-body interaction, and levitation-based displays. Most of these applications share two key limitations: a) the complexity of the sound fields that can be produced is limited by the physical size of the transducers, and b) no obstacles can be present between the transducers and the control point. We present SoundBender, a hybrid system that overcomes these limitations by combining the versatility of phased arrays of Transducers (PATs) with the precision of acoustic metamaterials. In this paper, we explain our approach to design and implement such hybrid modulators (i.e. to create complex sound fields) and methods to manipulate the field dynamically (i.e. stretch, steer). We demonstrate our concept using self-bending beams enabling both levitation and tactile feedback around an obstacle and present example applications enabled by SoundBender.

The ability to control acoustic fields offers many possible applications in loudspeaker design, ultrasound imaging, medical therapy, and acoustic levitation. Sound waves are currently shaped using phased array systems, even though the complex electronics required are expensive and hinder widespread use. Here we show how to control, direct, and manipulate sound using 2-dimensional, planar, acoustic metasurfaces that require only one driving signal. This offers the advantages of ease of use and versatility over currently available phased arrays. We demonstrate the creation of a haptic sensation and steering of a beam produced by a parametric speaker. This simple, yet highly effective, method of creating single-beam manipulators could be introduced in medical or manufacturing applications.

In this paper, we demonstrate a method for the dynamic 2D transformation of liquid matter and present unique organic animations based on spatio-temporally controlled electric fields. In particular, we deploy a droplet of liquid metal (Gallium indium eutectic alloy) in a 7x7 electrode array prototype system, featuring an integrated image tracking system and a simple GUI. Exploiting the strong dependance of EGaIn's surface tension on external electric voltages, we control multiple electrodes dynamically to manipulate the liquid metal into a fine-grained desired shape. Taking advantage of the high conductivity of liquid metals, we introduce a shape changing, reconfigurable smart circuit as an example of unique applications. We discuss system constraints and the overarching challenge of controlling liquid metals in the presence of phenomena such as splitting, self-electrode interference and finger instabilities. Finally, we reflect on the broader vision of this project and discuss our work in the context of the wider scope of programmable materials.

We present a programmable liquid matter which can dynamically transform its 2D shape into a variety of forms and present unique organic animations based on spatio-temporally controlled electric fields. We deployed a EGaIn (Gallium indium eutectic alloy) liquid metal as our smart liquid material since it features a superior electric conductivity in spite of a liquid state and presents a high dynamic range of surface tension and 2D area controlled by applied voltage strength and polarity. Our proposed liquid metal shape and motion control algorithms with dynamically patterned electric fields realize path tracing organic animation. We demonstrate an interactive 7x7 electrode array control system with a computer vision based GUI system to enable novice users to physically draw alphabet letters and 2D shapes by unique animatronics of liquid metals.

We present two realizations of TastyFloats, a novel system that uses acoustic levitation to deliver food morsels to the users' tongue. To explore TastyFloats' associated design framework, we first address the technical challenges to successfully levitate and deliver different types of foods on the tongue. We then conduct a user study, assessing the effect of acoustic levitation on users' taste perception, comparing three basic taste stimuli (i.e., sweet, bitter and umami) and three volume sizes of droplets (5μL, 10μL and 20μL). Our results show that users perceive sweet and umami easily, even in minimal quantities, whereas bitter is the least detectable taste, despite its typical association with an unpleasant taste experience. Our results are a first step towards the creation of new culinary experiences and innovative gustatory interfaces.