Here, we present the outcome of the 4th workshop on Multisensory Approaches to Human-Food Interaction (MHFI), developed in collaboration with ICMI 2020 in Utrecht, The Netherlands. Capitalizing on the increasing interest on multisensory aspects of human-food interaction and the unique contribution that our community offers, we developed a space to discuss ideas ranging from mechanisms of multisensory food perception, through multisensory technologies, to new applications of systems in the context of MHFI. All in all, the workshop involved 11 contributions, which will hopefully further help shape the basis of a field of inquiry that grows as we see progress in our understanding of the senses and the development of new technologies in the context of food.

The excitement around computing technology in all aspects of life requires that we tackle fundamental issues of healthcare, leisure, labor, education, and food to create the society we want. The aim of this satellite event was to bring together a variety of different stakeholders, ranging from local food producers, chefs, designers, engineers, data scientists, and sensory scientists, to discuss the interwoven future of computing technology and food. This event was co-located with the AVI 2018 conference and supported by the ACM Future of Computing Academy (ACM-FCA). The event followed a co-creation approach that encourages conjoined creative and critical thinking that feeds into the formulation of a manifesto on the future of computing and food. We hope this will inspire future discussions on the transformative role of computing technology on food.

This is the introduction paper to the third version of the workshop on 'Multisensory Approaches to Human-Food Interaction' organized at the 20th ACM International Conference on Multimodal Interaction in Boulder, Colorado, on October 16th, 2018. This workshop is a space where the fast growing research on Multisensory Human-Food Interaction is presented. Here we summarize the workshop's key objectives and contributions.

We present an introduction paper to the second version of the workshop on ‘Multisensory Approaches to Human-Food Interaction’ to be held at the 19th ACM International Conference on Multimodal Interaction, which will take place on November 13th, 2017, in Glasgow, Scotland. Here, we describe the workshop’s objectives, the key contributions of the different position papers, and the relevance of their respective topic(s). Both multisensory research and new technologies are evolving fast which has opened up a number of possibilities for designing new ways of interacting with foods and drinks. This workshop highlights the rapidly growing interest in the field of Multisensory Human-Food Interaction, which can, for example, be observed in the variety of novel research and technology developments in the area.

The use of the senses of vision and audition as interactive means has dominated the field of Human-Computer Interaction (HCI) for decades, even though nature has provided us with many more senses for perceiving and interacting with the world around us. That said, it has become attractive for HCI researchers and designers to harness touch, taste, and smell in interactive tasks and experience design. In this paper, we present research and design insights gained throughout an interdisciplinary collaboration on a six-week multisensory display – Tate Sensorium – exhibited at the Tate Britain art gallery in London, UK. This is a unique and first time case study on how to design art experiences whilst considering all the senses (i.e., vision, sound, touch, smell, and taste), in particular touch, which we exploited by capitalizing on a novel haptic technology, namely, mid-air haptics. We first describe the overall set up of Tate Sensorium and then move on to describing in detail the design process of the mid-air haptic feedback and its integration with sound for the Full Stop painting by John Latham (1961). This was the first time that mid-air haptic technology was used in a museum context over a prolonged period of time and integrated with sound to enhance the experience of visual art. As part of an interdisciplinary team of curators, sensory designers, sound artists, we selected a total of three variations of the mid-air haptic experience (i.e., haptic patterns), which were alternated at dedicated times throughout the six-week exhibition. We collected questionnaire-based feedback from 2500 visitors and conducted 50 interviews to gain quantitative and qualitative insights on visitors’ experiences and emotional reactions. Whilst the questionnaire results are generally very positive with only a small variation of the visitors’ arousal ratings across the three tactile experiences designed for the Full Stop painting, the interview data shed light on the differences in the visitors’ subjective experiences. Our findings suggest multisensory designers and art curators can ensure a balance between surprising experiences versus the possibility of free exploration for visitors. In addition, participants expressed that experiencing art with the combination of mid-air haptic and sound was immersive and provided an up-lifting experience of touching without touch. We are convinced that the insights gained from this large-scale and real-world field exploration of multisensory experience design exploiting a new and emerging technology provide a solid starting point for the HCI community, creative industries, and art curators to think beyond conventional art experiences. Specifically, our work demonstrates how novel mid-air technology can make art more emotionally engaging and stimulating, especially abstract art that is often open to interpretation.

This is an introductory paper for the workshop entitled ‘Multi-Sensorial Approaches to Human-Food Interaction’ held at ICMI 2016, which took place the 16th of November, 2016 in Tokyo, Japan. Here we discuss our objectives and the relevance of the workshop, and summarize the key contributions of the position papers. We were able to gather a group of researchers from different countries in Europe and Asia who presented their research and discussed the current developments, trends, limitations, and future applications of the field. Whilst this is the first workshop of its kind, we anticipate that the field of multisensory Human-Food Interaction (HFI) will grow in the upcoming years in terms of research and development, and its products will impact our everyday eating experiences.

This workshop aims to increase awareness of the different ways in which new technologies (e.g. social media, virtual reality, digital, and neuroimaging technologies) can be used as input or means to develop multisensory experiences that improve consumer satisfaction. Consumers experience products and services via their senses (i.e. sight, hearing, smell, taste, touch). The specific multisensory content of those products and services can influence people's respective attitudes, purchase intentions, and consumption. Today, consumers experiment the world through new technologies, for example, by sharing their feelings via social media, or by creating avatar in a virtual world. The challenge for marketing consists of monitoring new technologies to better understand consumer sensory experiences and to develop innovative actions that will help to improve these experiences. We will begin with the definition of sensory marketing and the role of multisensory processes in consumer experience. Second, we will address the possibility of better understanding the consumer through social media and neuroimaging technologies. To finish, we will present some technologies that are already enriching sensory experiences of consumers.

The development of neuroimaging techniques such as functional magnetic resonance imaging (fMRI) has provided marketers with the possibility of studying changes in brain activity in relation to marketing information (packaging, pricing, promotions, etc.) and decision-making. When consumers interact with a product, the brain captures and stores the related sensory experience in a multisensory representation in memory. Such memories serve consumers when making their future decisions. Thus, the senses play an important role in consumer behavior and the results of fMRI studies provide an interesting way for sensory marketers to analyze how the brain processes sensory information affecting product perception. In this article, we show how multisensory information, as well as mental simulation, impact taste expectations and subsequently taste, or better-said, flavor, perception in the light of neuroimaging studies. First, we discuss how neural analysis can help to understand the effect of marketing information on taste perception. Second, we highlight the role of gustatory inference on food perception in light of the latest fMRI findings. Third, we suggest some directions to improve the understanding of crossmodal correspondences in the context of consumer sensory experiences.

We present rich descriptions of taste experience through an analysis of the diachronic and synchronic experiences of each of the five basic taste qualities: sweet, sour, salt, bitter, and umami. Our findings, based on a combination of user experience evaluation techniques highlight three main themes: temporality, affective reactions, and embodiment. We present the taste characteristics as a framework for design and discuss each taste in order to elucidate the design qualities of individual taste experiences. These findings add a semantic understanding of taste experiences, their temporality enhanced through descriptions of the affective reactions and embodiment that the five basic tastes elicit. These findings are discussed on the basis of established psychological and behavioral phenomena, highlighting the potential for taste-enhanced design.