Kening Zhu

dblp:33/7597 · DBLP profile ↗
← Back
41ranked-venue papers
11as first author
19since 2021 · last 2026
0000-0001-6740-4921ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 33 · 11 first-author · 12 since 2021Graphics, computer vision, multimedia, augmented reality and games · 13 · 2 first-author · 9 since 2021
YearPublicationVenuePosition
2026 AnkleType: A Hands- and Eyes-free Foot-based Text Entry Technique in Virtual Reality
abstract
Virtual Reality (VR) emphasizes immersive experiences, while text entry often requires hands or visual attention, which may disrupt the interaction flows in VR. We present AnkleType, a hand- and eye-free text-entry technique that leverages ankle-based gestures for both standing and sitting situations. We began with two preliminary studies: one investigated the movement range of users' ankles, and the other elicited user-preferred ankle gestures for text-entry-related operations. The findings of these two studies guided our design of AnkleType. To optimize AnkleType's keyboard layout for eye-free input, we conducted a user study to capture the users' natural ankle spatial awareness with a computer-simulated language test. Through a pairwise comparison study, we designed a bipedal input strategy for sitting (BPSit) and a unipedal input strategy for standing (UPStand). Our first in-VR text-entry evaluation with 16 participants demonstrated that our methods could support the average typing speed from 8.99 WPM (BPSit) to 9.13 WPM (UPStand)for our first-time users. We further evaluated our design with a 7-day longitudinal study with twelve participants. Participants achieved an average typing speed of 15.05 WPM with UPStand BPSit in the visual condition, and 11.15 WPM and 12.87 WPM, respectively in the eyes-free condition. © 2026 Copyright held by the owner/author(s).
Xiyun Luo, Weirong Luo, Kening Zhu, Taizhou Chen
CHI3
2026 Bondi: Designing Tangible and Multimodal Interfaces for Continuing Bonds in Pet Bereavement
abstract
Pet ownership creates profound human-animal bonds, making pet loss significant. However, compared to human loss, Human-Computer Interaction (HCI) has given pet loss less attention. Addressing this, we conducted an exploratory mixed-methods study. The study began with a large-scale survey (N=611) revealing critical challenges: a strong desire to preserve memories but limited support for continuing bonds. Built upon these findings, our participatory design sessions(N=10) co-designed Bondi, a tangible prototype supporting continuing bonds through multimodal and customizable interactions (e.g. touch, sound, and light), evoking pets’ unique sounds, tail movements, and lighting effects. We then conducted a three-week field-deployment study with four participants to evaluate how Bondi facilitated the maintenance of bonds with their deceased pets. Results showed that the customization and multimodality evoked vivid recollections, lowering the social barrier for grief sharing. Bondi fostered comforting and non-intrusive connections with pet memories. Furthermore, the study distilled design considerations for future pet bereavement support.
Ningchang Xiong, Yanheng Li 0002, Man Chong Chin, Kening Zhu
CHI4
2026 LiqMetCraft: A Toolkit for Creating Papercraft with Embedded Electronics By Directly Cutting and Folding Liquid-Metal-Dyed Paper-like Fabric
abstract
Creating interactive papercrafts often involves the processes of craft making (e.g., folding, cutting, gluing, etc.) and fabricating the embedded functional circuitry. These two processes are usually separated in the current practice, potentially making the workflow laborious and affecting the in-paper circuit stability. To address the issue of the separated crafting and circuiting processes, we present LiqMetCraft, a toolkit for creating electronics-embedded papercrafts through an integrated process. The toolkit allows users to construct the craft structure by folding and cutting, and these crafting actions forms the circuit traces simultaneously. This is achieved by using liquid-metal-dyed paper-like fabric which partially becomes conductive due to the cutting/folding-induced pressure while the unpressed parts of the paper remain insulated. The LiqMetCraft toolkit consists of software interfaces for papercraft design and instructions, and hardware components, mainly the liquid-metal-dyed paper-like fabrics and other off-the-shelf components, for physical prototyping. Currently our toolkit supports the making of electronics-embedded paper cutting, origami, and 3D paper models. The user studies with 42 participants shows that the participants quickly learned the toolkit and found the integrated process of circuit assembly and shape formation to be engaging and inspiring.
Qi Zhang 0111, Shuwen Jiang, Zeshui Li, Yong Lyu, Tiande Mo, Kening Zhu
CHI6
2026 ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning
abstract
Providing haptic feedback in virtual reality (VR) remains challenging, with active haptic systems often being bulky and expensive, while passive haptics typically require pre-fabricated objects or extensive real-world setup. We present ProXeek, an LLM-based multi-agent system that opportunistically leverages real-world objects as haptic proxies for VR experiences. During VR development, the designers could annotate the virtual interactables with emphasized haptic characteristics using our node-based editing interface. On the user side, end users capture environmental snapshots via commodity VR headset, and the system employs LLM-based multimodal reasoning and multi-objective optimization to match virtual objects with viable physical proxies. Our technical evaluation demonstrates that ProXeek effectively identifies viable proxies, confirming the effectiveness of the algorithmic constraint design in our matching pipeline. Our user study further reveals that the ProXeek-selected physical objects could deliver significantly higher haptic fidelity and enhance presence compared to 3D-printed replicas and visual-only interaction.
Haichen Gao, Tianrui Hu, Zeshui Li, Kening Zhu
ACM Trans. Graph.4
2025 ThreadTessel: A Modularized Tangible Toolkit Leveraging Origami Tessellation for Designing Thread-actuated Shape-changing Structures
Lina Zhang 0007, Yibin Huai, Zhian Hu, Jiayi Wu 0016, Álvaro Cassinelli, Kening Zhu
TEI6
2025 ViboPneumo: A Vibratory-Pneumatic Finger-Worn Haptic Device for Altering Perceived Texture Roughness in Mixed Reality
abstract
Extensive research has been done in haptic feedback for texture simulation in virtual reality (VR). However, it is challenging to modify the perceived tactile texture of existing physical objects which usually serve as anchors for virtual objects in mixed reality (MR). In this article, we present ViboPneumo, a finger-worn haptic device that uses vibratory-pneumatic feedback to modulate (i.e., increase and decrease) the perceived roughness of the material surface contacted by the user's fingerpad while supporting the perceived sensation of other haptic properties (e.g., temperature or stickiness) in MR. Our device includes a silicone-based pneumatic actuator that can lift the user's fingerpad on the physical surface to reduce the contact area for roughness decreasing, and an on-finger vibrator for roughness increasing. The results of our perceptual study showed that the participants could perceive changes in roughness, both increasing and decreasing, compared to the original material surface. We also observed the overlapping roughness ratings among certain haptic stimuli (i.e., vibrotactile and pneumatic) and the originally perceived roughness of some materials without any haptic feedback. This suggests the potential to alter the perceived texture of one type of material to another in terms of roughness (e.g., modifying the perceived texture of ceramics as glass). Lastly, a user study of MR experience showed that ViboPneumo could significantly improve the MR user experience, particularly for visual-haptic matching, compared to the condition of a bare finger. We also demonstrated a few application scenarios for ViboPneumo.
Shaoyu Cai, Zhenlin Chen, Haichen Gao, Qi Zhang 0111, Xinge Yu, Kening Zhu
IEEE Trans. Vis. Comput. Graph.7
2024 Emotion Embodied: Unveiling the Expressive Potential of Single-Hand Gestures
abstract
Hand gestures are widely used in daily life for expressing emotions, yet gesture input is not part of existing emotion tracking systems. To seek a practical and effortless way of using gestures to inform emotions, we explore the relationships between gestural features and commonly experienced emotions by focusing on single-hand gestures that are easy to perform and capture. First, we collected 756 gestures (in photo and video pairs) from 63 participants who expressed different emotions in a survey, and then interviewed 11 of them to understand their gesture-forming rationales. We found that the valence and arousal level of the expressed emotions significantly correlated with participants’ finger-pointing direction and their gesture strength, and synthesized four channels through which participants externalized their expressions with gestures. Reflecting on the findings, we discuss how emotions can be characterized and contextualized with gestural cues and implications for designing multimodal emotion tracking systems and beyond.
Yuhan Luo 0002, Junnan Yu, Minhui Liang, Yichen Wan, Kening Zhu, Shannon Sie Santosa
CHI5
2024 ThermOuch: A Wearable Thermo-Haptic Device for Inducing Pain Sensation in Virtual Reality through Thermal Grill Illusion
abstract
Existing research showed that unpleasant haptic feedback, such as pain, could enhance the user experience and performance in various scenarios (e.g.entertainment and training).This paper introduces ThermOuch, a wearable thermo-haptic device that leverages the thermal grill illusion (TGI) to simulate pain sensations in virtual reality (VR) without causing actual invasive/non-invasive harm.Our results of the user-perception experiments revealed that higher temperature-changing rates, particularly with increased warming, were associated with more intense pain perceived by the participants through our system.Furthermore, a higher ratio of warm-to-cool temperature transitions reduced the sensation of coldness prior to pain.Our experiments also showed that introducing an additional stimulus unit potentially heightened pain perception, and altering the spacing between stimulus units modified the perceived pain area.Lastly, the user study in VR demonstrated that ThermOuch significantly enhanced the sense of presence and body ownership for the participants, as well as elevated their biosignal-indicated arousal levels.
Haichen Gao, Shaoyu Cai, Yuhong Wu, Kening Zhu
SIGGRAPH Asia4
2024 WhisperCup: A Design Exploration for Improving the Remote Communication between Chinese Parents and Their Adult Children through Digitally-Augmented Everyday Objects
abstract
Chinese families usually place a strong emphasis on maintaining a sense of integrality even after the children have grown up. However, this might pose challenges in remote communication. For instance, over-frequent remote communication may disturb each other's life; and because of Chinese people's conservative way of expression, pervasive communication tools, such as video or voice calls, cannot communicate each other's emotions conveniently. To enhance remote communication between Chinese parents and their adult children, we employed a three-stage design process to identify the target users' needs in remote communication and presented WhisperCup. Our study indicated that WhisperCup could unintentionally increase daily communication between Chinese parents and adult children. The awareness system embedded in the WhisperCup prototype could provide a glimpse of each other's daily life, helping virtually engage in each other's life. Additionally, we found that WhisperCup could facilitate a better understanding of each other's lives while addressing privacy concerns.
Lina Zhang 0007, Jiaan Li, Álvaro Cassinelli, Kening Zhu
Proc. ACM Hum. Comput. Interact.5
2024 PetPresence: Investigating the Integration of Real-World Pet Activities in Virtual Reality
abstract
For VR interaction, the home environment with complicated spatial setup and dynamics may hinder the VR user experience. In particular, pets' movement may be more unpredictable. In this paper, we investigate the integration of real-world pet activities into immersive VR interaction. Our pilot study showed that the active pet movements, especially dogs, could negatively impact users' performance and experience in immersive VR. We proposed three different types of pet integration, namely semitransparent real-world portal, non-interactive object in VR, and interactive object in VR. We conducted the user study with 16 pet owners and their pets. The results showed that compared to the baseline condition without any pet-integration technique, the approach of integrating the pet as interactive objects in VR yielded significantly higher participant ratings in perceived realism, joy, multisensory engagement, and connection with their pets in VR.
Ningchang Xiong, Qingqin Liu, Kening Zhu
IEEE Trans. Vis. Comput. Graph.3
2023 Emoband: Investigating the Affective Perception towards On-wrist Stroking and Squeezing Feedback mediated by Different Textile Materials
abstract
This paper investigates how different textile materials on the wrist bands may mediate the affective experience of two forms of tactile icons, namely stroking and squeezing. To this end, we designed Emoband, a wrist-worn textile-based tactile display that can stroke or squeeze on wearers’ wrists through the moving fabrics. We conducted two studies on the participants’ valence and arousal ratings towards the stroking and squeezing feedback from Emoband, respectively. Results showed that the valence ratings were significantly affected by the stroking or squeezing parameters and the wristband’s material, interactively. The movement-related parameters dominated the arousal rating. Meanwhile, the valence ratings demonstrated different tendencies between the two tactile feedback, with the valence increasing and decreasing along with the strength of stroking and squeezing, respectively. Besides, the valence ratings of the stroking stimuli demonstrated varied distributions across different materials, indicating the materials’ varying affective expressiveness in the wrist-worn haptic devices.
Kening Zhu
CHI2
2023 Deep-learning-based unobtrusive handedness prediction for one-handed smartphone interaction
Taizhou Chen, Kening Zhu, Ming-Chieh Yang
Multim. Tools Appl.2
2023 PropelWalker: A Leg-Based Wearable System With Propeller-Based Force Feedback for Walking in Fluids in VR
abstract
There has been an increasing focus on haptic interfaces for virtual reality (VR), to support a high-quality touch experience. However, it is still challenging to haptically simulate the real-world walking experience in different fluid mediums. To tackle this problem, we present PropelWalker, a pair of calf-worn haptic devices for simulating the buoyancy and the resistant force when the human's lower limbs are interacting with different fluids and materials in VR. By using four ducted fans, two installed on each calf, the system can control the strength and the direction of the airflow in real time to provide different levels of force. Our technical evaluation shows that PropelWalker can generate vertical forces up to 27N in two directions (i.e., upward and downward) within 0.85 seconds. Furthermore, the system can stably maintain the generated force with minor turbulence. We further conducted three user-perception studies to understand the capability of PropelWalker to generate distinguishable force stimuli. First, we conducted the just-noticeable-difference (JND) experiments to investigate the threshold of the human perception of on-leg air-flow force feedback. Our second perception study showed that users could distinguish four PropelWalker-generated force levels for simulating different walking mediums (i.e., dry ground, water, mud, and sand), with an average accuracy of 94.2%. Lastly, our VR user study showed that PropelWalker could significantly improve the users' sense of presence in VR.
Pingchuan Ke, Shaoyu Cai, Haichen Gao, Kening Zhu
IEEE Trans. Vis. Comput. Graph.4
2022 GAN-based image-to-friction generation for tactile simulation of fabric material
Shaoyu Cai, Yuki Ban, Takuji Narumi, Yue Liu 0005, Kening Zhu
Comput. Graph.6
2022 Understanding and Adapting Bezel-to-Bezel Interactions for Circular Smartwatches in Mobile and Encumbered Scenarios
abstract
Supporting eyes-free interaction, mobility and encumbrance, while providing a broad set of commands on a smartwatch display is a difficult, yet important, task. Bezel-to-bezel (B2B) gestures are valuable for rapid command invocation during eyes-free operation, however we lack knowledge regarding B2B interactions on circular devices during common usage scenarios. We aim to improve our understanding of the dynamics of B2B interactions in these scenarios by conducting two studies and a third analysis: First, we explore the performance of B2B in a seated position; second, we explore the effect of mobility and encumbrance on the B2B interaction; finally, we improve on the B2B accuracies by calculating features and utilizing machine learning. With the limited interaction capabilities on smartwatches and the importance of the scenario of use, we conclude with applications and design guidelines for improved utilization of B2B that enables effective smartwatch control while in common, mobile and eyes-free scenarios.
Bradley Rey, Kening Zhu, Simon T. Perrault, Sandra Bardot, Ali Neshati, Pourang Irani
Proc. ACM Hum. Comput. Interact.2
2021 ThermEarhook: Investigating Spatial Thermal Haptic Feedback on the Auricular Skin Area
abstract
Haptic feedbacks are widely adopted in mobile and wearable devices to convey various types of notifications to the users. This paper investigates the design and the evaluation of thermal haptic feedback on an earable form factor with multiple thermoelectric (i.e. Peltier) modules. We propose ThermEarhook, a wearable device that can provide hot and cold stimuli at multiple points on the auricular skin area. To investigate users’ thermal perception on the auricular area, we develop a series of ThermEarhook prototypes with 3, 4, and 5 Peltier modules. While most existing research utilized the constant level of haptic signal for different users, our pilot study with ThermEarhook shows that the auricular thermohaptic threshold varies across the feedback locations and the users. With the user-customized thermohaptic signals around the ear, our first study with 12 participants reports on the selection of the auricular configuration with four TEC modules on each side, considering the users’ identification accuracy (averagely 99.3%) and preference. We then conduct three follow-up studies and a total of 36 participants to further evaluate users’ perception of spatial thermal patterns with ThermEarhook, and finalize a set of multi-points auricular thermal patterns that can be reliably perceived by the users with the average accuracy of 85.3%. Lastly, we discuss the user-proposed potential applications of the thermal haptic feedback with ThermEarhook.
Arshad Nasser, Kexin Zheng, Kening Zhu
ICMI3
2021 Larger Step Faster Speed: Investigating Gesture-Amplitude-based Locomotion in Place with Different Virtual Walking Speed in Virtual Reality
abstract
In this paper, we present a series of user studies to investigate the technique of gesture-amplitude-based walking-speed control for locomotion in place (LIP) in virtual reality (VR). Our 1st study suggested that compared to tapping and goose-stepping, the gesture of marching in place was significantly preferred by users across three different virtual walking speed (i.e., 1 ×, 3 ×, and 10 ×) while sitting and standing, and it yielded larger motion difference across the three speed levels. With the tracker data recorded in the 1st study, we trained a Support- Vector-Machine classification model for LIP speed control based on users' leg/foot gestures in marching in place. The overall accuracy for classifying three speed levels was above 90% for sitting and standing. With the classification model, we then compared the marching-in-place speed-control technique with the controller-based teleportation approach on a target-reaching task where users were sitting and standing. We found no significant difference between the two conditions in terms of target-reaching accuracy. More importantly, the technique of marching in place yielded significantly higher user ratings in terms of naturalness, realness, and engagement than the controller-based teleportation did.
Pingchuan Ke, Kening Zhu
VR2
2021 FritzBot: A data-driven conversational agent for physical-computing system design
Taizhou Chen, Lantian Xu 0001, Kening Zhu
Int. J. Hum. Comput. Stud.3
2021 GestOnHMD: Enabling Gesture-based Interaction on Low-cost VR Head-Mounted Display
abstract
Low-cost virtual-reality (VR) head-mounted displays (HMDs) with the integration of smartphones have brought the immersive VR to the masses, and increased the ubiquity of VR. However, these systems are often limited by their poor interactivity. In this paper, we present GestOnHMD, a gesture-based interaction technique and a gesture-classification pipeline that leverages the stereo microphones in a commodity smartphone to detect the tapping and the scratching gestures on the front, the left, and the right surfaces on a mobile VR headset. Taking the Google Cardboard as our focused headset, we first conducted a gesture-elicitation study to generate 150 user-defined gestures with 50 on each surface. We then selected 15, 9, and 9 gestures for the front, the left, and the right surfaces respectively based on user preferences and signal detectability. We constructed a data set containing the acoustic signals of 18 users performing these on-surface gestures, and trained the deep-learning classification pipeline for gesture detection and recognition. Lastly, with the real-time demonstration of GestOnHMD, we conducted a series of online participatory-design sessions to collect a set of user-defined gesture-referent mappings that could potentially benefit from GestOnHMD.
Taizhou Chen, Lantian Xu 0001, Xianshan Xu, Kening Zhu
IEEE Trans. Vis. Comput. Graph.4
2020 ThermalCane: Exploring Thermotactile Directional Cues on Cane-Grip for Non-Visual Navigation
abstract
Non-visual feedback (e.g., auditory and haptic) has been used as directional cues for the blind and visually impaired (BVI) users. This paper presents the design and the evaluation of ThermalCane, a white-cane grip instrumented with multiple flexible thermal modules, to offer thermotactile directional cues for BVI users. We also conducted two thermotactile experiments on users’ perception of ThermalCane. Our first experiment with twelve BVI users reports on the selection of the thermal-module configuration, considering the BVI users’ perceptive accuracy and preference. We then evaluated the effectiveness of the four-module ThermalCane in walking with 6 BVI users, in comparison with vibrotactile cues. The results show that the thermal feedback yielded significantly higher accuracy than the vibrotactile feedback. The results also suggested the feasibility of using thermal directional cues around the cane grip for BVI users’ navigation.
Arshad Nasser, Kai-Ning Keng, Kening Zhu
ASSETS3
2020 Exploring Eyes-free Bezel-initiated Swipe on Round Smartwatches
abstract
Bezel-based gestures expand the interaction space of touch-screen devices (e.g., smartphones and smartwatches). Existing works have mainly focused on bezel-initiated swipe (BIS) on square screens. To investigate the usability of BIS on round smartwatches, we design six different circular bezel layouts, by dividing the bezel into 6, 8, 12, 16, 24, and 32 segments. We evaluate the user performance of BIS on these layouts in an eyes-free situation. The results show that the performance of BIS is highly orientation dependent, and varies significantly among users. Using the Support-Vector-Machine (SVM) model significantly increases the accuracy on 6-, 8-, 12-, and 16-segment layouts. We then compare the performance of personal and general SVM models, and find that personal models significantly improve the accuracy for 8-, 12-, 16-, and 24-segment layouts. Lastly, we discuss the potential smartwatch applications enabled by the BIS.
Pui Chung Wong, Kening Zhu, Xing-Dong Yang, Hongbo Fu 0001
CHI2
2020 ThermAirGlove: A Pneumatic Glove for Thermal Perception and Material Identification in Virtual Reality
abstract
We present ThermAirGlove (TAGlove), a pneumatic glove which provides thermal feedback for users, to support the haptic experience of grabbing objects of different temperatures and materials in virtual reality (VR). The system consists of a glove with five inflatable airbags on the fingers and the palm, two temperature chambers (one hot and one cold), and the closed-loop pneumatic thermal control system. Our technical experiments showed that the highest temperature-changing speed of TAGlove system was 2.75°C/s for cooling, and the pneumatic-control mechanism could generate the thermal cues of different materials (e.g., foam, glass, copper, etc.). The user-perception experiments showed that the TAGlove system could provide five distinct levels of thermal sensation (ranging from very cool to very warm). The user-perception experiments also showed that the TAGlove could support users’ material identification among foam, glass, and copper with the average accuracy of 87.2%, with no significant difference compared to perceiving the real physical objects. The user studies on VR experience showed that using TAGlove in immersive VR could significantly improve users’ experience of presence compared to the situations without any temperature or material simulation.
Shaoyu Cai, Pingchuan Ke, Takuji Narumi, Kening Zhu
VR4
2019 Thermo-haptic Earable Display for the Hearing and Visually Impaired
abstract
Earables/hearables (ear-worn devices) are on the rise as next-generation wearables which are capable of streaming audio, modifying soundscapes, and collecting body vitals. This paper introduces a novel concept of an earable device that can provide thermal (hot and cold) haptic cues at multiple areas around the ear. The concept with discrete thermal encoding can help the hearing and visually impaired individuals to obtain directional and other notifications from mobile and IoT devices. Embedding thermal haptic feedback into an earable form factor have a better edge in terms of privacy of notifications when compared to audio, visual, and vibrohaptic modalities. We conducted a pilot study using a proof-of-concept prototype and confirmed the feasibility of the concept.
Arshad Nasser, Kening Zhu, Sarah Wiseman
ASSETS2
2019 HapTwist: Creating Interactive Haptic Proxies in Virtual Reality Using Low-cost Twistable Artefacts
abstract
In this paper, we present a series of studies on using Rubik's Twist, a type of low-cost twistable artefact, to create haptic proxies for various hand-graspable VR objects. Our pilot studies validated the feasibility and effectiveness of Rubik's-Twist-based haptic proxies. The pilot results also revealed user challenges in the physical shape creation, motivating the development of the HapTwist toolkit. The toolkit consists of the shape-generation algorithm, the software interface for shape-construction guidance and interaction authoring, and the hardware modules for constructing interactive haptic proxies. The user studies showed that HapTwist was easy to learn and use, and it significantly improved user performance in creating interactive haptic proxies with Rubik's Twist. Furthermore, HapTwist-generated haptic proxies achieved similar VR experience as the real objects.
Kening Zhu, Taizhou Chen, Feng Han 0004, Yi-Shiun Wu
CHI1
2019 DupRobo: Interactive Robotic Autocompletion of Physical Block-Based Repetitive Structure
Taizhou Chen, Yi-Shiun Wu, Kening Zhu
INTERACT (2)3
2019 Additive Voronoi Cursor: Dynamic Effective Areas Using Additively Weighted Voronoi Diagrams
Jacky Kit Cheung, Oscar Kin-Chung Au, Kening Zhu
INTERACT (3)3
2019 TipText: Eyes-Free Text Entry on a Fingertip Keyboard
abstract
In this paper, we propose and investigate a new text entry technique using micro thumb-tip gestures. Our technique features a miniature QWERTY keyboard residing invisibly on the first segment of the user's index finger. Text entry can be carried out using the thumb-tip to tap the tip of the index finger. The keyboard layout was optimized for eyes-free input by utilizing a spatial model reflecting the users' natural spatial awareness of key locations on the index finger. We present our approach of designing and optimizing the keyboard layout through a series of user studies and computer simulated text entry tests over 1,146,484 possibilities in the design space. The outcome is a 2×3 grid with the letters highly confining to the alphabetic and spatial arrangement of QWERTY. Our user evaluation showed that participants achieved an average text entry speed of 11.9 WPM and were able to type as fast as 13.3 WPM towards the end of the experiment.
Zheer Xu, Pui Chung Wong, Jun Gong 0002, Te-Yen Wu, Aditya Shekhar Nittala, Xiaojun Bi 0001, Jürgen Steimle, Hongbo Fu 0001, Kening Zhu, Xing-Dong Yang
UIST9
2019 A sense of ice and fire: Exploring thermal feedback with multiple thermoelectric-cooling elements on a smart ring
Kening Zhu, Simon T. Perrault, Taizhou Chen, Shaoyu Cai, Roshan Lalintha Peiris
Int. J. Hum. Comput. Stud.1
2018 WristOrigami: Exploring Foldable Design for Multi-Display Smartwatch
abstract
We present WristOrigami, an origami-inspired design concept and system extending the interaction with smartwatches through a foldable structure with multiple on-wrist displays. The current design provides extra affordances via folding, flipping, and elastic pulling actions on a multi-display smartwatch. To motivate the design of WristOrigami, we developed a taxonomy that could be useful for analyzing and characterizing the origami-inspired multi-display smartwatch interaction. Through a participatory-design study with a set of prototypes with different levels of fidelity, we investigated users' perception of WristOrigami in a wide range of applications with the presented features, and summarized a list of common shape configurations. We summarized our findings into seven design recommendations, to inform the future design of foldable smartwatch interactions. We further developed a set of application demonstrations as proofs-of-concept.
Kening Zhu, Morten Fjeld, Adviye Ayça Ünlüer
Conference on Designing Interactive Systems1
2018 FingerT9: Leveraging Thumb-to-finger Interaction for Same-side-hand Text Entry on Smartwatches
abstract
We introduce FingerT9, leveraging the action of thumb-to-finger touching on the finger segments, to support same-side-hand (SSH) text entry on smartwatches. This is achieved by mapping a T9 keyboard layout to the finger segments. Our solution avoids the problems of fat finger and screen occlusion, and enables text entry using the same-side hand which wears the watch. In the pilot study, we determined the layout mapping preferred by the users. We conducted an experiment to compare the text-entry performances of FingerT9, the tilt-based SSH input, and the direct-touch non-SSH input. The results showed that the participants performed significantly faster and more accurately with FingerT9 than the tilt-based method. There was no significant difference between FingerT9 and direct-touch methods in terms of efficiency and error rate. We then conducted the second experiment to study the learning curve on SSH text entry methods: FingerT9 and the tilt-based input. FingerT9 gave significantly better long-term improvement. In addition, eyes-free text entry (i.e., looking at the screen output but not the keyboard layout mapped on the finger segments) was made possible once the participants were familiar with the keyboard layout.
Pui Chung Wong, Kening Zhu, Hongbo Fu 0001
CHI2
2018 Investigating different modalities of directional cues for multi-task visual-searching scenario in virtual reality
abstract
In this study, we investigated and compared the effectiveness of visual, auditory, and vibrotactile directional cues on multiple simultaneous visual-searching tasks in an immersive virtual environment. Effectiveness was determined by the task-completion time, the range of head movement, the accuracy of the identification task, and the perceived workload. Our experiment showed that the on-head vibrotactile display can effectively guide users towards virtual visual targets, without affecting their performance on the other simultaneous tasks, in the immersive VR environment. These results can be applied to numerous applications (e.g. gaming, driving, and piloting) in which there are usually multiple simultaneous tasks, and the user experience and performance could be vulnerable.
Taizhou Chen, Yi-Shiun Wu, Kening Zhu
VRST3
2017 Teaching and Learning of Chinese History in Minecraft: A Pilot Case-Study in Hong Kong Secondary Schools
abstract
Minecraft is the best-selling and the most popular digital sandbox games, allowing children to freely create their own buildings and objects. It has been widely used in education. In this research, we developed a real-scale Forbidden City in Minecraft, as a history education platform for Hong Kong secondary schools. The platform allows teachers to manage the class in Minecraft, meanwhile preserving the freedom for students to explore, collaborate, and create. We piloted the platform with 3 teachers and 50 students, and collected feedbacks on the teaching and the learning experience, for future development.
Kening Zhu, John Man Ho Huen
IDC1
2017 Mirrortablet: exploring a low-cost mobile system for capturing unmediated hand gestures in remote collaboration
abstract
Direct and natural images of hand gestures have been shown to benefit remote collaboration on physical tasks in several settings, including ad-hoc ones. However, to capture such unmediated hand gestures within collaborative tasks, existing approaches require stationary hardware systems or heavily instrumented mobile devices, making them unfeasible for use at ad-hoc workplaces. We present MirrorTablet, a low-cost mirror-based system taking advantage of the built-in front-facing camera of a tablet to capture the user's unmediated hand interactions on and above the screen. This system requires minimal instrumentation of the tablet and can be easily (un-)mounted, making it suitable for mobile usage. A user study with ten pairs of participants on a helper-worker setup working on construction tasks showed that MirrorTablet improved task completion time and had positive effects on participants' perceived workload when working with unfamiliar tasks compared to using a common sketch-only interface. In addition, qualitative feedback yielded design considerations on hand visualization for mobile device remote collaboration on physical tasks.
Khanh-Duy Le, Kening Zhu, Morten Fjeld
MUM2
2017 Tripartite Effects: Exploring Users' Mental Model of Mobile Gestures under the Influence of Operation, Handheld Posture, and Interaction Space
abstract
In this research, we conducted an elicitation study with 54 participants to evaluate user-defined mobile gestures given three types of operations, handheld postures, and interaction spaces, respectively, to investigate the effects of these factors on mental model. The results of the elicitation study revealed that each of the three factors significantly affected the users’ mental models, which could be reflected by the characteristics (e.g., nature, number of steps, and spatial utility) of the user-defined mobile gestures. In addition, we found that users tended to assign different roles to mid-air and on-screen gestures based on their motor ability given a handheld gesture and their interpretation of the characteristics and requirements of an operation. We further derived a set of gestures commonly defined in the elicitation study for a list of representative operations, and we share our insights into users’ mental models and the implications on future mobile-gesture design in different contexts.
Kening Zhu, Xiaojuan Ma, Miaoyin Liang
Int. J. Hum. Comput. Interact.1
2016 FusePrint: A DIY 2.5D Printing Technique Embracing Everyday Artifacts
abstract
FusePrint is a Stereolithography-based 2.5D rapid prototyping technique that allows high-precision fabrication without high-end modeling tools, enabling the mixing of everyday physical artifacts and liquid conductive gels with photo-reactive resin during the printing process, facilitating the creation of 2.5D objects that perfectly fit the existing objects. Based on our polynomial model on 2.5D resin printing, we developed the design interface of FusePrint, which allows users to design the printed shapes using physical objects as references, generates projection patterns, and notifies users when to place the objects in the resin during the printing process. Our workshops suggested that FusePrint is easy to learn and use, provides a greater level of interactivity, and could be useful for a wide range of applications domains including: mechanical fabrication, wearable accessory, toys, interactive systems, etc.
Kening Zhu, Alexandru Dancu, Shengdong Zhao 0001
Conference on Designing Interactive Systems1
2016 How Different Input and Output Modalities Support Coding as a Problem-Solving Process for Children
abstract
Using coding education to promote computational thinking and nurture problem-solving skills in children has become an emerging global trend. However, how different input and output modalities in coding tools affect coding as a problem-solving process remains unclear. Of interest are the advantages and disadvantages of graphical and tangible interfaces for teaching coding to children. We conducted four kids coding workshops to study how different input and output methods in coding affected the problem-solving process and class dynamics. Results revealed that graphical input could keep children focused on problem solving better than tangible input, but it was less provocative for class discussion. Tangible output supported better schema construction and casual reasoning and promoted more active class engagement than graphical output but offered less affordance for analogical comparison among problems. We also derived insights for designing new tools and teaching methods for kids coding.
Kening Zhu, Xiaojuan Ma, Gary K. W. Wong, John Man Ho Huen
IDC1
2016 The Development of Internationalized Computational Thinking Curriculum in Hong Kong Primary Education (Abstract Only)
abstract
Computational Thinking (CT) has been widely introduced and investigated in recent years, particularly in the U.S. since the born of visual, block-based, drag-drop programming environments such as Kodu, Scratch, Minecraft and App Inventor. Although the user interface is mainly in English, the characteristics of these easy-to-use, game-based, and interactive tools attract many teachers and researchers in the world to pay much attention to the possibilities and opportunities of introducing these tools to students. Recently, some primary school teachers in Hong Kong begin to independently introduce some of these programming tools to students at age 7 - 11 as a part of learning activities in their computer lessons. Their motives are similar but not the same, such as making a fun learning and teaching experience, motivating students for active and collaborative participation, and introducing CT concepts to develop generic skills (e.g. problem solving skills, creativity, and critical thinking). However, there is an absence of well-developed and planned curriculum for "coding education" to introduce computational thinking systematically to students in the local context with expected learning outcomes. Due to the uniqueness of K-12 curriculum in Hong Kong, the existing curriculum model in the U.S. may need to be customized and redesigned to become suitable for integrating into the curriculum in Hong Kong. In this poster, it describes the first proposed coding education curriculum in Hong Kong primary education (Primary 4 to Primary 6) with relevant objectives, structures, contents, and learning outcomes. A new pedagogical design framework for CT is introduced in this poster, which could be generalizable and yet to be evaluated. This new curriculum will serve as the curriculum guide to local teachers, and is the first research initiative of a three-year longitudinal study investigating the impact of CT activities to students particularly in Hong Kong. The experience of this curriculum development for CT concepts in K-12 education can inspire teachers and researchers in other parts of the world when adopting and internationalizing CT activities based on the curriculum model developed under the U.S. education.
Gary K. W. Wong, Kening Zhu, Xiaojuan Ma, John Man Ho Huen
SIGCSE2
2013 AutoGami: a low-cost rapid prototyping toolkit for automated movable paper craft
abstract
AutoGami is a toolkit for designing automated movable paper craft using the technology of selective inductive power transmission. AutoGami has hardware and software components that allow users to design and implement automated movable paper craft without any prerequisite knowledge of electronics; it also supports rapid prototyping. Apart from developing the toolkit, we have analyzed the design space of movable paper craft and developed a taxonomy to facilitate the design of automated paper craft. AutoGami made consistently strong showings in design workshops, confirming its viability in supporting engagement and creativity as well as its usability in storytelling through paper craft. Additional highlights include rapid prototyping of product design as well as interaction design such as human-robot interactions.
Kening Zhu, Shengdong Zhao 0001
CHI1
2013 Designing Interactive Paper-Craft Systems with Selective Inductive Power Transmission
abstract
Paper, as a traditional material for art and communication, shows great potential as a medium for organic user interfaces, with its ubiquity and flexibility. However, controlling and powering the sensors and actuators that enable interactive paper-crafts has not been fully explored. We present a method of selective inductive power transmission (SIPT) to support interactive paper-crafts. The novelty of this method is that the power transmitter can be controlled to selectively activate one specific receiver at a time through inductive power transferring with multiple receivers. This was achieved by changing the output frequency of the power transmitter to match the impedance of the receivers. The receivers could be embedded or printed to drive paper-crafts. Based on inductor–capacitor oscillating circuit and a function generator with a power amplifier, we developed two different prototypes of SIPT. By comparing the performance of both prototypes, we discussed the advantages and disadvantages of the two systems, and their applications in different contexts of paper-crafts. In addition, we proposed the instructions for using SIPT in developing interactive paper-crafts. With this technology and instructions, we hope to facilitate users to easily design new types of paper-craft systems without being concerned about the arrangement of wire connections to power supply on a massive scale.
Kening Zhu, Hideaki Nii, Owen Noel Newton Fernando, Jeffrey Tzu Kwan Valino Koh, Karin Aue, Adrian David Cheok
Interact. Comput.1
2010 Origami recognition system using natural feature tracking
abstract
This paper introduces a system that can recognize different type of paper-folding by users. The system allows users to register and use their desired paper in the interaction, and detect the folding by using Speed Up Robust Feature (SURF) algorithm. The paper also describes a paper-based tower defense game which has been developed as a proof of concept of our method. This method can be considered as the initial step for seamlessly migrating meaningful traditional art of origami into the digital world as a part of the interactive media.
Kening Zhu, Owen Noel Newton Fernando, Adrian David Cheok, Mark Fiala, Theam Wei Yang
ISMAR1
2010 Origami recognition system using natural feature tracking
abstract
In this demonstration we introduce a system that allows users to register and use their desired paper for the interaction, and recognize the folding during the interaction in real time. This method can be considered as an initial step for seamlessly migrating meaningful traditional art of origami into the digital world and as part of the interactive media.
Kening Zhu, Owen Noel Newton Fernando, Theam Wei Yang, Adrian David Cheok, Mark Fiala
ISMAR1