Jialin Deng

dblp:271/9884 · DBLP profile ↗
← Back
15ranked-venue papers
5as first author
14since 2021 · last 2026
—ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 11 · 5 first-author · 11 since 2021Computer networks · 4 · 3 since 2021
YearPublicationVenuePosition
2026 Composing for the Palate: Designing and Investigating Taste-Matching Sounds for Cross-Sensory Interaction
abstract
Sound has long been used to enhance eating experiences, and increasingly, we know sound can shape how we perceive taste. Yet intentionally composing sounds to modulate taste remains underexplored in cross-sensory HCI design. While prior research has mapped individual auditory parameter (e.g., pitch, timbre, rhythm) to basic taste perceptions, less is known about how holistic sound compositions combining multiple parameters can be systematically designed and evaluated. We present a human-in-the-loop workflow that brings together professional sound designers and generative AI to create 160 sounds with taste intentions informed by established auditory-taste parameters. In a study with 149 participants on sound-taste correspondences, we identify sounds that strongly correspond to basic taste words. Our contributions are threefold: (1) a workflow for designing taste-matching sounds; (2) a curated repository of reusable sound stimuli for cross-sensory research; and (3) a designerly framework for creating compositional taste-matching sounds alongside insights into sound-taste word correspondences and association strategies, offering implications for cross-sensory HCI.
Jialin Deng, Min Susan Li, Zhuzhi Fan, Hongyue Wang 0001, Peter Bennett 0001, Ziqi Fang, Priscilla Lo, Oussama Metatla
DIS1
2026 triMorph: Bridging Shape-Change and Cross-Sensory Correspondences for Haptic Interaction
abstract
Cross-sensory correspondences provide opportunities for designing rich sensory HCI, with prior work showing that features such as roundness and sharpness are systematically linked to language, color, sound, and emotion. Yet two challenges remain: few technologies can dynamically transition between these features, and little is known about the thresholds at which a form is judged as sufficiently rounded or spiky to realize these cross-sensory effects. We present triMorph, a pneumatic shape-changing interface capable of smoothly morphing between spiky, flat, and rounded configurations. In a psychophysical study with 30 participants, we quantified perceptual accuracy and precision in mapping triMorph shapes to visual-linguistic categories and examined shape–color and shape–emotion correspondences. Results reveal threshold values for reliable categorization, with rounded shapes linked to pleasant emotions and lighter colors, and spiky shapes to arousal and darker tones. Our findings provide empirical foundations and design guidelines for grounding shape-changing artifacts more firmly in cross-sensory cognition.
Zhuzhi Fan, Min Susan Li, Jialin Deng, Amy Ingold, Oussama Metatla
CHI3
2026 GastroConcerto: Towards Designing Dining-Sound Pairings to Support Culinary Creativity
abstract
Sounds play a crucial role in shaping dining experiences. Recently, designers have increasingly integrated them into interfaces that connect diners with food. Yet, little is known about how sounds can function as culinary materials to enrich chefs’ creativity, particularly in creating meaningful auditory interactions that resonate with their culinary creations. Hence, we present GastroConcerto, an auditory dining system that combines a magnetic contact microphone equipped under the plate with a companion mobile application. This system delves into the interactive space between diners and their food to introduce a novel interactive mechanism of dining–sound pairing, enabling chefs to design specific auditory interactions that respond to diners’ individual interactions with the food on dining containers. Through a within-subjects study and field deployment, we examined how GastroConcerto enriches chefs’ creative practices in crafting "sonic dish" experiences. Ultimately, our goal is to shift the ownership of auditory interaction design from interface designers to chefs, thereby supporting their culinary creativity.
Hongyue Wang 0001, Sasindu Abewickrema, Yuchen Zheng 0002, Po-Yao (Cosmos) Wang, Ziqi Fang, Jialin Deng, Nandini Pasumarthy, Samitha Elvitigala, Florian 'Floyd' Mueller
CHI7
2025 LuciEntry: Towards Understanding the Design of Lucid Dream Induction
abstract
DIS ’25, Funchal, Portugal
Po-Yao (Cosmos) Wang, Zoe Xiao Fang, Gabriel Ducos, Nathaniel Yung Xiang Lee, Antony Smith Loose, Rohit Rajesh, Nethmini Botheju, Maria Fernanda Montoya, Alexandra Kitson, Karen Konkoly, Rohan Sagi, Rakesh Patibanda, Nathan W. Whitmore, Mahdad Jafarzadeh Esfahani, Jialin Deng, Jiajun Bu, Martin Dresler, Samitha Elvitigala, Nathan Arthur Semertzidis, Florian 'Floyd' Mueller
Conference on Designing Interactive Systems16
2025 Sonic Delights: Exploring the Design of Food as An Auditory-Gustatory Interface
abstract
While interest in blending sound with culinary experiences has grown in Human-Food Interaction (HFI), the significance of food's material properties in shaping sound-related interactions has largely been overlooked. This paper explores the opportunity to enrich the HFI experience by treating food not merely as passive nourishment but as an integral material in computational architecture with input/output capabilities. We introduce “Sonic Delights,” where food is a comestible auditory-gustatory interface to enable users to interact with and consume digital sound. This concept redefines food as a conduit for interactive auditory engagement, shedding light on the untapped multisensory possibilities of merging taste with digital sound. An associated study allowed us to articulate design insights for forthcoming HFI endeavors that seek to weave food into multisensory design, aiming to further the integration of digital interactivity with the culinary arts.
Jialin Deng, Yinyi Li, Hongyue Wang 0001, Ziqi Fang, Florian 'Floyd' Mueller
CHI1
2025 Towards Understanding Interactive Sonic Gastronomy with Chefs and Diners
Hongyue Wang 0001, Jialin Deng, Linjia He, Nathalie Overdevest, Ryan Wee, Yan Wang 0057, Phoebe O. Toups Dugas, Samitha Elvitigala, Florian 'Floyd' Mueller
CHI2
2024 From Plating to Tasting: Towards Understanding the Choreography of Computational Food
abstract
The emerging concept of “computational food” focusing on the material affordances when designing food interactions is gaining traction in Human-Food Interaction (HFI). However, prior HFI research has not yet substantively investigated the dynamic nature of computational food from its creation to consumption, limiting our understanding of the complex interactions among creators, computational food, and consumers. In response, our paper shifts the perspective towards the dynamics of computational food interactions through a study in cooperation with chefs and gastronomists. Utilizing “Dancing Delicacies” as a research artifact – a system that facilitates dynamic dining trajectories – we adopted the concept of “choreography” to unravel the experiential dynamics of computational food. Our study resulted in six themes concerning computational food experiences and detailed four design implications central to culinary choreography. Our work aspires to leverage the choreographic potential of computational food design, paving the way for future HFI innovations.
Jialin Deng, Nathalie Overdevest, Patrick Olivier, Florian 'Floyd' Mueller
CHI1
2024 FreAuth: Novel Frequency Feature-Based Device Authentication for Magnetic Wireless Charging
abstract
Device authentication plays a crucial role in preventing illegal access and ensuring smooth usage of magnetic wireless charging. However, current authentication techniques suffer from security vulnerabilities and are incompatible with low-cost receiver devices, thus severely limiting their applications. In this paper, we propose FreAuth, a novel Frequency feature-based device Authentication technology for magnetic wireless power transfer systems. Technically, we begin by conducting circuit measurements at the transmitter side to subtly retrieve impedance information related to the receiver without the need for its corporation. Then, we employ a dual-frequency interleaved-based subtraction technique to remove the ideal receiver impedance and capture the fairly weak frequency features. Furthermore, we normalize the captured frequency features to account for environment variations. These steps allow us to generate and store a hardware fingerprint for the receiver based on its frequency features. During device authentication, we use a discrete Frechet distance-based algorithm for fingerprint matching. We devise and implement a prototype of FreAuth and conduct extensive experiments to evaluate the proposed scheme. The experimental results validate the reliability (95.74% authentication accuracy among 60+ devices) and robustness (anti-interference with device location variations) of our FreAuth.
Shenyao Jiang, Wangqiu Zhou, Hao Zhou 0001, Jialin Deng, Haisheng Tan, Zhi Liu 0002, Zhenjiang Li 0001
IWQoS4
2024 Grand challenges in human-food interaction
abstract
There is an increasing interest in combining interactive technology with food, leading to a new research area called human-food interaction. While food experiences are increasingly benefiting from interactive technology, for example in the form of food tracking apps, 3D-printed food and projections on dining tables, a more systematic advancement of the field is hindered because, so far, there is no comprehensive articulation of the grand challenges the field is facing. To further and consolidate conversations around this topic, we invited 21 HFI experts to a 5-day seminar. The goal was to review our own and prior work to identify the grand challenges in human-food interaction. The result is an articulation of 10 grand challenges in human-food interaction across 4 categories (technology, users, design and ethics). By presenting these grand challenges, we aim to help researchers move the human-food interaction research field forward.
Florian 'Floyd' Mueller, Marianna Obrist, Ferran Altarriba Bertran, Neharika Makam, Sohyeong Kim, Christopher Dawes, Patrizia Marti, Maurizio Mancini, Eleonora Ceccaldi, Nandini Pasumarthy, Sahej Claire, Kyung seo Jung, Jialin Deng, Jürgen Steimle, Nadejda Krasteva, Matti Schwalk, Harald Reiterer, Hongyue Wang 0001, Yan Wang 0057
Int. J. Hum. Comput. Stud.13
2023 Dancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories
abstract
Contemporary human-food interaction design is often a technology-driven endeavor in which food's materiality has been largely underexplored. Building on the concept of “computational food”, this paper explores the design of food as a material realization of computation through a material-centered approach. We engaged with a “Research through Design” exploration by designing a computational food system called “Dancing Delicacies”, which enables food items to be “programmed” and “reconfigured” within dynamic trajectories. Our practice led to a design framework resulting in four original dish designs. Our dishes aim to illustrate the richness of this new design space for computational food. Furthermore, through engaging with expert practitioners from the hospitality industry, we provide a first account of understanding the design of computational food for dynamic dining trajectories and its speculative use contexts in the industry. With this work, we hope to inspire researchers and designers to envision a new future of human-food interaction.
Jialin Deng, Humphrey Yang, Aryan Saini, Urs Dominic Gaudenz, Lining Yao, Patrick Olivier, Florian 'Floyd' Mueller
Conference on Designing Interactive Systems1
2023 PROCS: Power Routing and Current Scheduling in Multi-Relay Magnetic MIMO WPT System
abstract
Magnetic resonant coupling wireless power transfer (MRC-WPT) enables convenient device-charging. When MIMO MRC-WPT system incorporated with multiple relay components, both relayOn-Offstate (i.e.,power routing) and TX current (i.e.,current scheduling) could be adjusted for improving charging efficiency and distance. Previous approaches need the collaboration and feedback from the energy receiver (RX), achieved using side-channels, e.g., Bluetooth, which is time/energy-consuming. In this work we propose, design, and implement a multi-relay MIMO MRC-WPT system, and design an almost optimum joint optimization ofPowerROuting andCurrentScheduling method namedPROCS, without relying on any feedback from RX. We carefully decompose the joint optimization problem into two subproblems without affecting the overall optimality of the combined solution. For current scheduling subproblem, we propose an almost-optimum RX-feedback independent solution. For power routing subproblem, we first design a greedy algorithm with$\frac{1}{2}$approximation ratio, and then design a DQN based method to further improve its effectiveness. We prototype our system and evaluate it with extensive experiments. Our results demonstrate the effectiveness of the proposed algorithms. The achieved power transfer efficiency (PTE) on average is$3.2X$,$1.43X$,$1.34X$, and$7.3X$over the other four strategies: Without relay, with non-adjustable relays, greed based, and shortest-path based ones.
Hao Zhou 0001, Jialin Deng, Wenxiong Hua, Xiang Cui, Xiang-Yang Li 0001, Panlong Yang
IEEE Trans. Mob. Comput.2
2022 Logic Bonbon: Exploring Food as Computational Artifact
abstract
In recognition of food's significant experiential pleasures, culinary practitioners and designers are increasingly exploring novel combinations of computing technologies and food. However, despite much creative endeavors, proposals and prototypes have so far largely maintained a traditional divide, treating food and technology as separate entities. In contrast, we present a “Research through Design” exploration of the notion of food as computational artifact: wherein food itself is the material of computation. We describe the Logic Bonbon, a dessert that can hydrodynamically regulate its flavor via a fluidic logic system. Through a study of experiencing the Logic Bonbon and reflection on our design practice, we offer a provisional account of how food as computational artifact can mediate new interactions through a novel approach to food-computation integration, that promotes an enriched future of Human-Food Interaction.
Jialin Deng, Patrick Olivier, Josh Andres, Kirsten Ellis, Ryan Wee, Florian 'Floyd' Mueller
CHI1
2022 Mag-E4E: Trade Efficiency for Energy in Magnetic MIMO Wireless Power Transfer System
abstract
Magnetic resonant coupling (MRC) wireless power transfer (WPT) is a convenient and potential power supply solution for smart devices. The scheduling problem in the multiple-input multiple-output (MIMO) scenarios is essential to concentrate energy at the receiver (RX) side. Meanwhile, strong TX-RX coupling could ensure better power transfer efficiency (PTE), but may cause lower power delivered to load (PDL) when transmitter voltages are bounded. In this paper, we propose the frequency adjustment based PDL maximization scheme for MIMO MRC-WPT systems. We formulate such joint optimization problem and decouple it into two sub-problems, i.e., high-level frequency adjustment and low-level voltage adaptation. We solve these two sub-problems with gradient descent based and alternating direction method of multipliers (ADMM) based algorithms, respectively. We further design an energy-voltage transform matrix algebra based estimation mechanism to reduce context measurement overhead. We prototype the proposed system, and conduct extensive experiments to evaluate its performance. As compared with the PTE maximization solutions, our system trades smaller efficiency for larger energy, i.e., 361% PDL improvement with respect to 26% PTE losses when TX-RX distance is 10cm.
Xiang Cui, Hao Zhou 0001, Jialin Deng, Wangqiu Zhou, Yu Gu 0003
INFOCOM3
2021 Data as Delight: Eating data
abstract
The HCI community has a rich history of finding new ways to engage people with data beyond the screen. With our work, we aim to expand the scope of how interaction design can engage people, arguing that “eating data” has the potential to allow people to experience “data as delight”. With reference to prior work and our design research findings, we discuss the advantages and the challenges of this approach to integrating data and food. We then identify four themes to guide the design of engagements with data through food: food form, food commensality, food ephemerality, and emotional response to food. Within these design themes, we articulate twelve insights for interaction designers to use when working on serving data as delight.
Florian 'Floyd' Mueller, Tim Dwyer, Sarah Goodwin, Kim Marriott, Jialin Deng, Han Duy Phan, Jionghao Lin, Kun-Ting Chen, Yan Wang 0057, Rohit Ashok Khot
CHI5
2020 Joint Power Routing and Current Scheduling in Multi-Relay Magnetic MIMO WPT System
abstract
Magnetic resonant coupling wireless power transfer (MRC-WPT) enables convenient device-charging. When MIMO MRC-WPT system incorporated with multiple relay components, both relay On-Off state (i.e., power routing) and TX current (i.e., current scheduling) could be adjusted for improving charging efficiency and distance. Previous approaches need the collaboration and feedback from the energy receiver (RX), achieved using side-channels, e.g., Bluetooth, which is time/energy-consuming. In this work we propose, design, and implement a multi-relay MIMO MRC-WPT system, and design an almost optimum joint optimization of power routing and current scheduling method, without relying on any feedback from RX. We carefully decompose the joint optimization problem into two subproblems without affecting the overall optimality of the combined solution. For current scheduling subproblem, we propose an almost-optimum RX-feedback independent solution. For power routing subproblem, we first design a greedy algorithm with ½ approximation ratio, and then design a DQN based method to further improve its effectiveness. We prototype our system and evaluate it with extensive experiments. Our results demonstrate the effectiveness of the proposed algorithms. The achieved power transfer efficiency (PTE) on average is 3.2X, 1.43X, 1.34X, and 7.3X over the other four strategies: without relay, with non-adjustable relays, greed based, and shortest-path based ones.
Hao Zhou 0001, Wenxiong Hua, Jialin Deng, Xiang Cui, Xiang-Yang Li 0001, Panlong Yang
INFOCOM3