The tongue is a remarkable human organ with a high concentration of taste receptors and an exceptional ability to sense touch. This work uses electro-tactile stimulation to explore the intricate interplay between tactile perception and taste rendering on the tongue. To facilitate this exploration, we utilized a flexible, high-resolution electro-tactile prototyping platform that can be administered in the mouth. We have created a design tool that abstracts users from the low-level stimulation parameters, enabling them to focus on higher-level design objectives. Through this platform, we present the results of three studies. Our first study evaluates the design tool’s qualitative and formative aspects. In contrast, the second study measures the qualitative attributes of the sensations produced by our device, including tactile sensations and taste. In the third study, we demonstrate the ability of our device to sense touch input through the tongue when placed on the hard palate region in the mouth. Finally, we present a range of application demonstrators that span diverse domains, including accessibility, medical surgeries, and extended reality. These demonstrators showcase the versatility and potential of our platform, highlighting its ability to enable researchers and practitioners to explore new ways of leveraging the tongue’s unique capabilities. Overall, this work presents new opportunities to deploy tongue interfaces and has broad implications for designing interfaces that incorporate the tongue as a sensory organ.

People are typically involved in different activities while eating, particularly when eating alone, such as watching television or playing games on their phones. Previous research in Human-Food Interaction (HFI) has primarily focused on studying people’s motivation and analyzing of the media content watched while eating. However, their impact on human behavioral and cognitive processes, particularly flavor perception and its attributes, remains underexplored. We present a user study to investigate the influence of six types of videos, including mukbang – a new food video genre, on flavor perceptions (taste sensations, liking, and emotions) while eating plain white rice. Our findings revealed that participants perceived positive emotional changes and reported significant differences in their augmented taste sensations (e.g., spicy and salty) with different food-based videos. Our findings provided insights into using our approach to promote digital commensality and healthier eating (digital augmentation without altering the food), highlighting the scope for future research.

Little is known about the influence of various sensory modalities such as taste, smell, color, and thermal, towards perceiving simulated flavor sensations, let alone their influence on people's emotions and liking. Although flavor sensations are essential in our daily experiences and closely associated with our memories and emotions, the concept of flavor and the emotions caused by different sensory modalities are not thoroughly integrated into Virtual and Augmented Reality technologies. Hence, this paper presents 1) an interactive technology to simulate different flavor sensations by overlaying taste (via electrical stimulation on the tongue), smell (via micro air pumps), color (via RGB Lights), and thermal (via Peltier elements) sensations on plain water, and 2) a set of experiments to investigate a) the influence of different sensory modalities on the perception and liking of virtual flavors and b) varying emotions mediated through virtual flavor sensations. Our findings reveal that the participants perceived and liked various stimuli configurations and mostly associated them with positive emotions while highlighting important avenues for future research.

Wind simulations are typically one-off implementations for specific applications. We introduce WindyWall, a platform for creative design and exploration of wind simulations. WindyWall is a three-panel 90-fan array that encapsulates users with 270? of wind coverage. We describe the design and implementation of the array panels, discussing how the panels can be re-arranged, where various wind simulations can be realized as simple effects. To understand how people perceive "wind" generated from WindyWall, we conducted a pilot study of wind magnitude perception using different wind activation patterns from WindyWall. Our findings suggest that: horizontal wind activations are perceived more readily than vertical ones, and that people's perceptions of wind are highly variable-most individuals will rate airflow differently in subsequent exposures. Based on our findings, we discuss the importance of developing a method for characterizing wind simulations, and provide design directions for others using fan arrays to simulate wind.

In the same way that we experience the real-world through a range of senses, experiencing a virtual environment through multiple sensory modalities may augment both our presence within a scenario and our reaction to it. In this paper, we present Season Traveller, a multisensory virtual reality (VR) narration of a journey through four seasons within a mystical realm. By adding olfactory and haptic (thermal and wind) stimuli, we extend traditional audio-visual VR technologies to achieve enhanced sensory engagement within interactive experiences. Using both subjective measures of presence and elicited physiological responses, we evaluated the impact of different modalities on the virtual experience. Our results indicate that 1) the addition of any singular modality improves sense of presence with respect to traditional audio-visual experiences and 2) providing a combination of these modalities produces a further significant enhancement over the aforementioned improvements. Furthermore, insights into participants' psychophysiology were extrapolated from electrodermal activity (EDA) and heart rate (HR) measurements during each of the VR experiences.

AltMM 2018 is the 3rd edition of the International Workshop on Multimedia Alternate Realities at ACM Multimedia. Our ambition remains to engage researchers and practitioners in discussions on how we can successfully create meaningful multimedia 'alternate realities' experiences. One of the main strengths of this workshop is that we combine different perspectives to explore how the synergy between multimedia technologies can foster and shape the creation of alternate realities and make their access an enriching and valuable experience.

This demonstration presents ``Tainted', a first-person 3D adventure game that enables players to experience a traditional horror folklore using visual-auditory-olfactory interactions. The system consists of a computer to play the game and a smell-emitting module that contains jasmine, pineapple, mango, and banana scents. The information provided by the additional smell stimuli delivers an immersive gaming experience by providing game narrative and signaling predefined game conditions (e.g. imminent threat). Smell stimuli may also help to enhance player's cognition and memory, as well as to invoke emotional responses such as fear and curiosity.

In this paper, we present and evaluate Ambiotherm, a wearable accessory for Head Mounted Displays (HMD) that provides thermal and wind stimuli to simulate real-world environmental conditions, such as ambient temperatures and wind conditions, to enhance the sense of presence in Virtual Reality (VR). Ambiotherm consists of a Ambient Temperature Module that is attached to the user's neck, a Wind Simulation Module focused towards the user's face, and a Control Module utilizing Bluetooth communication. We demonstrate Ambiotherm with two VR environments, a hot desert, and a snowy mountain, to showcase the different types of simulated environmental conditions. We conduct several studies to 1) address design factors of the system and 2) evaluate Ambiotherm's effect on factors related to a user's sense of presence. Our findings show that the addition of wind and thermal stimuli significantly improves sensory and realism factors, contributing towards an enhanced sense of presence when compared to traditional VR experiences.

Similar to the concept of a cocktail or mocktail, we present Vocktail (a.k.a. Virtual Cocktail) - an interactive drinking utensil that digitally simulates multisensory flavor experiences. The Vocktail system utilizes three common sensory modalities, taste, smell, and visual (color), to create virtual flavors and augment the existing flavors of a beverage. The system is coupled with a mobile application that enables users to create customized virtual flavor sensations by configuring each of the stimuli via Bluetooth. The system consists of a cocktail glass that is seamlessly fused into a 3D printed structure, which holds the electronic control module, three scent cartridges, and three micro air-pumps. When a user drinks from the system, the visual (RGB light projected on the beverage), taste (electrical stimulation at the tip of the tongue), and smell stimuli (emitted by micro air-pumps) are combined to create a virtual flavor sensation, thus altering the flavor of the beverage. In summary, this paper discusses 1) technical details of the Vocktail system and 2) user experiments that investigate the influences of these multimodal stimuli on the perception of virtual flavors in terms of five primary tastes (i.e. salty, sweet, bitter, sour, and umami). Our results suggest that the combination of these stimuli delivers richer flavor experiences, as compared to separately simulating individual modalities, and indicates that the types of pairings that can be formed between smell and electric taste stimuli.

This paper presents a novel methodology to digitally share the flavor experience of a glass of lemonade (or other similar beverages) remotely. The proposed method utilizes a sensor to capture valuable information (primarily, the color and the corresponding pH value) of the lemonade and a customized tumbler to virtually simulate these properties using plain water. Thus, the system consists of three main components: 1) the lemonade sensor, 2) the communication protocol, and 3) a customized tumbler, acting as the lemonade simulator. Initially, the sensor captures the color and the pH value of the lemonade and encodes this information based on an established communication protocol for wireless transmission. On receiving the information from the sensor, the lemonade simulator overlays the color of the drink on plain water using an RGB Light Emitting Diode (LED) and simulates sour taste sensations on the user's tongue via electrical stimulation. An experimental study was conducted to evaluate this novel approach of digitally teleporting a glass of lemonade: 1) to assess the pre-taste perceptions based on the user's visual perceptions of the colors (real vs. virtual lemonade) and 2) to assess the taste sensations (real vs. virtual lemonade). By simulating the experience of drinking a glass of lemonade through the digital reconstruction of the beverage's main visual and taste factors, the results from these experiments will be able to justify the feasibility of teleporting a glass of lemonade using this novel methodology.

Among the five primary senses, the sense of taste is the least explored as a form of digital media applied in Human--Computer Interface. This article presents an experimental instrument, the Digital Lollipop, for digitally simulating the sensation of taste (gustation) by utilizing electrical stimulation on the human tongue. The system is capable of manipulating the properties of electric currents (magnitude, frequency, and polarity) to formulate different stimuli. To evaluate the effectiveness of this method, the system was experimentally tested in two studies. The first experiment was conducted using separate regions of the human tongue to record occurrences of basic taste sensations and their respective intensity levels. The results indicate occurrences of sour, salty, bitter, and sweet sensations from different regions of the tongue. One of the major discoveries of this experiment was that the sweet taste emerges via an inverse-current mechanism, which deserves further research in the future. The second study was conducted to compare natural and artificial (virtual) sour taste sensations and examine the possibility of effectively controlling the artificial sour taste at three intensity levels (mild, medium, and strong). The proposed method is attractive since it does not require any chemical solutions and facilitates further research opportunities in several directions including human--computer interaction, virtual reality, food and beverage, as well as medicine.

Flavor is often a pleasurable sensory perception we experience daily while eating and drinking. However, the sensation of flavor is rarely considered in the age of digital communication mainly due to the unavailability of flavors as a digitally controllable media. This paper introduces a digital instrument (Digital Flavor Synthesizing device), which actuates taste (electrical and thermal stimulation) and smell sensations (controlled scent emitting) together to simulate different flavors digitally. A preliminary user experiment is conducted to study the effectiveness of this method with predefined five different flavor stimuli. Experimental results show that the users were effectively able to identify different flavors such as minty, spicy, and lemony. Moreover, we outline several challenges ahead along with future possibilities of this technology. In summary, our work demonstrates a novel controllable instrument for flavor simulation, which will be valuable in multimodal interactive systems for rendering virtual flavors digitally.

In recent years, digital media technologies expand horizons into non-traditional sensory modalities, for instance, the sense of taste. This demonstration presents `taste+,' which digitally improves the taste sensations of food and beverages without additional flavoring ingredients. It primarily utilizes weak and controlled electrical pulses on the human tongue to enhance sourness, saltiness, and bitterness of food or beverages. Taste+ consists of two prototype utensils, a bottle and a spoon. Both utensils have embedded electronic control modules to achieve enhanced taste sensations. These modules apply controlled electrical pulses on the tongue through silver electrodes attached to the mouth pieces. Furthermore, different superimposed colors symbolize distinct taste sensations, lemon green represents sour, ocean blue is salty, red for red wine bitter. The initial experimental results suggested that sourness and saltiness are the main sensations that could be evoked while bitterness has comparatively mild responses. Furthermore, we also observed several mixed sensations such as salty-sour together.

In this paper, we describe an interactive drinking platform, FunRasa, which digitally expands one's drinking experience. The system uses two methods: electrical stimulation on user's tongue and superimpose virtual color onto the actual drink, to expand the taste sensations of the drink. The system consists of a glass cup, an electronic platform with RGB LEDs, and a specially designed straw interface with two silver electrodes. When a user uses the straw, his tongue touches the silver electrodes (both lower and upper surfaces of the tip of the tongue) and is thus electrically stimulated when drinking. The user has the freedom to change the virtual color of the drink along with the magnitude of the current using two mechanical dials. Furthermore, we present several initial discussions on the user experience through a workshop that we conducted and several future usage scenarios of this technology.

In this paper, we present a new methodology for integrating the sense of taste with the existing digital communication domain. First, we discuss existing problems and limitations for integrating the sense of taste as a digital communication media. Then, to assess this gap, we present a solution with three core modules: the transmitter, form of communication, and receiver. The transmitter is a mobile application, where the sender formulates a taste message to send. Then, for communication, we present a new extensible markup language (XML) format, the TasteXML (TXML) to specify the format of taste messages. As the receiver (actuator), we introduce Digital Taste Stimulator, a novel method for stimulating taste sensations on human. Initial user experiments and qualitative feedbacks were discussed mainly focusing on the Digital Taste Stimulator. We conclude with a brief overview of future aspects of this technology and possibilities on other application domains.

In this paper, we present a system, which could digitally stimulate the sense of taste (gustation) on human. The system utilizes electrical stimulation on human tongue to produce taste sensations. The initial experiments reveal that the method is viable and deserves further developments. This requires further analyses of the properties of electric pulses (current, frequency, and voltage) on tongue along with the stimulating material. The experimental results suggested that sourness, bitterness, and saltiness are the main sensations that could be evoked at present.