Providing haptic feedback in virtual reality (VR) remains challenging, with active haptic systems often being bulky and expensive, while passive haptics typically require pre-fabricated objects or extensive real-world setup. We present ProXeek, an LLM-based multi-agent system that opportunistically leverages real-world objects as haptic proxies for VR experiences. During VR development, the designers could annotate the virtual interactables with emphasized haptic characteristics using our node-based editing interface. On the user side, end users capture environmental snapshots via commodity VR headset, and the system employs LLM-based multimodal reasoning and multi-objective optimization to match virtual objects with viable physical proxies. Our technical evaluation demonstrates that ProXeek effectively identifies viable proxies, confirming the effectiveness of the algorithmic constraint design in our matching pipeline. Our user study further reveals that the ProXeek-selected physical objects could deliver significantly higher haptic fidelity and enhance presence compared to 3D-printed replicas and visual-only interaction.

Despite being prominent and ubiquitous, message-based communication is limited in nonverbally conveying emotions. Besides emoticons or stickers, messaging users continue seeking richer options for affective communication. Recent research explored using chat-balloons’ shape and color to communicate emotional states . However, little work explored whether and how chat-balloon animations could be designed to convey emotions. We present the design of AniBalloons, 30 chat-balloon animations conveying Joy, Anger, Sadness, Surprise, Fear, and Calmness. Using AniBalloons as a research means, we conducted three studies to assess the animations’ affect recognizability and emotional properties ( N = 40 ), and probe how animated chat-balloons would influence communication experience in typical scenarios including instant messaging ( N = 72 ) and chatbot service ( N = 70 ). Our exploration contributes a set of chat-balloon animations to complement nonverbal affective communication for a range of text-message interfaces, and empirical insights into how animated chat-balloons might mediate particular conversation experiences (e.g., perceived interpersonal closeness, or chatbot personality).

Extensive research has been done in haptic feedback for texture simulation in virtual reality (VR). However, it is challenging to modify the perceived tactile texture of existing physical objects which usually serve as anchors for virtual objects in mixed reality (MR). In this article, we present ViboPneumo, a finger-worn haptic device that uses vibratory-pneumatic feedback to modulate (i.e., increase and decrease) the perceived roughness of the material surface contacted by the user's fingerpad while supporting the perceived sensation of other haptic properties (e.g., temperature or stickiness) in MR. Our device includes a silicone-based pneumatic actuator that can lift the user's fingerpad on the physical surface to reduce the contact area for roughness decreasing, and an on-finger vibrator for roughness increasing. The results of our perceptual study showed that the participants could perceive changes in roughness, both increasing and decreasing, compared to the original material surface. We also observed the overlapping roughness ratings among certain haptic stimuli (i.e., vibrotactile and pneumatic) and the originally perceived roughness of some materials without any haptic feedback. This suggests the potential to alter the perceived texture of one type of material to another in terms of roughness (e.g., modifying the perceived texture of ceramics as glass). Lastly, a user study of MR experience showed that ViboPneumo could significantly improve the MR user experience, particularly for visual-haptic matching, compared to the condition of a bare finger. We also demonstrated a few application scenarios for ViboPneumo.

Existing research showed that unpleasant haptic feedback, such as pain, could enhance the user experience and performance in various scenarios (e.g.entertainment and training).This paper introduces ThermOuch, a wearable thermo-haptic device that leverages the thermal grill illusion (TGI) to simulate pain sensations in virtual reality (VR) without causing actual invasive/non-invasive harm.Our results of the user-perception experiments revealed that higher temperature-changing rates, particularly with increased warming, were associated with more intense pain perceived by the participants through our system.Furthermore, a higher ratio of warm-to-cool temperature transitions reduced the sensation of coldness prior to pain.Our experiments also showed that introducing an additional stimulus unit potentially heightened pain perception, and altering the spacing between stimulus units modified the perceived pain area.Lastly, the user study in VR demonstrated that ThermOuch significantly enhanced the sense of presence and body ownership for the participants, as well as elevated their biosignal-indicated arousal levels.

There has been an increasing focus on haptic interfaces for virtual reality (VR), to support a high-quality touch experience. However, it is still challenging to haptically simulate the real-world walking experience in different fluid mediums. To tackle this problem, we present PropelWalker, a pair of calf-worn haptic devices for simulating the buoyancy and the resistant force when the human's lower limbs are interacting with different fluids and materials in VR. By using four ducted fans, two installed on each calf, the system can control the strength and the direction of the airflow in real time to provide different levels of force. Our technical evaluation shows that PropelWalker can generate vertical forces up to 27N in two directions (i.e., upward and downward) within 0.85 seconds. Furthermore, the system can stably maintain the generated force with minor turbulence. We further conducted three user-perception studies to understand the capability of PropelWalker to generate distinguishable force stimuli. First, we conducted the just-noticeable-difference (JND) experiments to investigate the threshold of the human perception of on-leg air-flow force feedback. Our second perception study showed that users could distinguish four PropelWalker-generated force levels for simulating different walking mediums (i.e., dry ground, water, mud, and sand), with an average accuracy of 94.2%. Lastly, our VR user study showed that PropelWalker could significantly improve the users' sense of presence in VR.