In our daily lives, we are exposed to various stressors, which, even in mild forms, can lead to serious health issues when prolonged. Dance Movement Therapy (DMT) uses bodily movement as psychotherapy to improve health and well-being; however, traditional DMT requires therapists, specific setups, and physical props, limiting daily accessibility. To address this, we conducted a formative study with 10 DMT experts and developed ImmersiveDMT, a VR application that provides real-time visual and auditory feedback in response to users’ movements. A user study with 56 participants examined how visual, auditory, and audiovisual feedback affect expressive movement and stress reduction compared to a non-responsive immersive baseline. Results showed that auditory and audiovisual conditions induced significantly more expressive movement compared to baseline. Both visual and auditory feedback achieved significant stress reduction below baseline (V: 17.1%, p=.004; A: 16.6%, p=.042), while combined audiovisual feedback led to less stress reduction. This research presents a novel approach to accessible daily stress management by integrating VR and DMT, demonstrating new possibilities for self-directed mental health support.

While commercial metaverse platforms offer diverse usergenerated content, they lack effective navigation assistance that can dynamically adapt to users' interests and intentions. Although previous research has investigated on-demand agents in controlled environments, implementation in commercial settings with diverse world configurations and platform constraints remains challenging. We present Navigation Pixie, an on-demand navigation agent employing a loosely coupled architecture that integrates structured spatial metadata with LLM-based natural language processing while minimizing platform dependencies, which enables experiments on the extensive user base of commercial metaverse platforms. Our cross-platform experiments on commercial metaverse platform Cluster with 99 PC client and 94 VR-HMD participants demonstrated that Navigation Pixie significantly increased dwell time and free exploration compared to fixed-route and noagent conditions across both platforms. Subjective evaluations revealed consistent on-demand preferences in PC environments versus context-dependent social perception advantages in VRHMD. This research contributes to advancing VR interaction design through conversational spatial navigation agents, establishes cross-platform evaluation methodologies revealing environment-dependent effectiveness, and demonstrates empirical experimentation frameworks for commercial metaverse platforms.

Gaze guidance is essential for directing user attention to specific areas of interest. However, conventional visual cues generate persistent visual noise that hinders concentration during tasks. We propose Saccaidance, a gaze-guidance method that appears only when users move their gaze. Saccaidance employs temporal additive color mixing and 480 Hz high-speed displays to shift the color phase of guidance patterns. This renders the patterns barely visible during fixation and makes them appear transiently when users move their gaze as a color-breaking effect. This intermittent gaze guidance appears only during gaze transitions, providing effective guidance without interfering with focused work or requiring eye-tracking hardware. We conducted experiments with 24 participants under four conditions that involved search tasks: an unmodified baseline, conventional explicit guidance, and our proposed method using oval and radial patterns. The results show that our approach effectively constrains the exploration area while preserving subjective naturalness. We also outline application scenarios of our method, including document highlighting.

Visual guidance (VG) plays an essential role in directing user attention in virtual reality (VR) and augmented reality (AR) environments. However, traditional approaches rely on explicit visual annotations, which often compromise visual clarity and increase user cognitive load. To address this issue, we propose an unobtrusive VG technique based on color vibration, a phenomenon in which rapidly alternating colors at frequencies above 25 Hz are perceived as a single intermediate color. Our work explores a perceptual state that exists between complete color fusion and visible flicker, where color differences remain detectable without conscious awareness of vibration. Through two experimental studies, we first identified the thresholds separating complete fusion, this intermediate perceptual state, and visible flicker by systematically varying color vibration parameters. Subsequently, we applied color vibrations with derived thresholds to natural image regions and validated their attention-guiding capabilities using eye-tracking measurements. The results demonstrate that controlled color vibration successfully directs user attention while maintaining low cognitive demand, providing an effective method for implementing unobtrusive VG in VR and AR systems.

The human brain’s plasticity allows for the integration of artificial body parts into the human body. Leveraging this, embodied systems realize intuitive interactions with the environment. We introduce a novel concept: embodied swarm robots. Swarm robots constitute a collective of robots working in harmony to achieve a common objective, in our case, serving as functional body parts. Embodied swarm robots can dynamically alter their shape, density, and the correspondences between body parts and individual robots. We contribute an investigation of the influence on embodiment of swarm robot-specific factors derived from these characteristics, focusing on a hand. Our paper is the first to examine these factors through virtual reality (VR) and real-world robot studies to provide essential design considerations and applications of embodied swarm robots. Through quantitative and qualitative analysis, we identified a system configuration to achieve the embodiment of swarm robots.

Robot swarms hold immense potential for performing complex tasks far beyond the capabilities of individual robots. However, the challenge in unleashing this potential is the robots’ limited sensory capabilities, which hinder their ability to detect and adapt to unknown obstacles in real-time. To overcome this limitation, we introduce a novel robot swarm control method with an indirect obstacle detector using a smoothed particle hydrodynamics (SPH) model. The indirect obstacle detector can predict the collision with an obstacle and its collision point solely from the robot’s velocity information. This approach enables the swarm to effectively and accurately navigate environments without the need for explicit obstacle detection, significantly enhancing their operational robustness and efficiency. Our method’s superiority is quantitatively validated through a comparative analysis, showcasing its significant navigation and pattern formation improvements under obstacle-unaware conditions.

Light Attenuation Displays (LADs), a subset of Optical See-Through Head-Mounted Displays (OST-HMDs), enable image display in bright environments by filtering incident light at the pixel level. Although recent methods have proposed single-DMD light attenuation, they do not consider additive color display and background compensation, limiting their applicability in real-world scenarios. We present FactoredSweeper, a single digital micromirror device (DMD) system that incorporates both light attenuation and addition. By synchronizing the DMD, color filter, and light source, our system generates an additive virtual image, light attenuation, and occlusion through time multiplexing. To produce the target image while compensating for the background, we optimize time-multiplexed binary DMD patterns and LED/color filter schedules using perceptually-driven non-negative matrix factorization. Simulations and prototypes demonstrate that our integrated attenuation-addition single-SLM system achieves superior dynamic range and perceptual image quality compared to conventional occlusion-capable OST-HMDs using grayscale occlusion masks.

Light Attenuation Displays (LADs) are a type of Optical See-Through Head-Mounted Display (OST-HMD) that present images by attenuating incoming light with a pixel-wise polarizing color filter. Although LADs can display images in bright environments, there is a trade-off between the number of Spatial Light Modulators (SLMs) and the color gamut and contrast that can be expressed, making it difficult to achieve both high-fidelity image display and a small form factor. To address this problem, we propose StainedSweeper, a LAD that achieves both the wide color gamut and the variable intensity with a single SLM. Our system synchronously controls a pixel-wise Digital Micromirror Device (DMD) and a nonpixel polarizing color filter to pass light when each pixel is the desired color. By sweeping this control at high speed, the human eye perceives images in a time-multiplexed, integrated manner. To achieve this, we develop the OST-HMD design using a reflective Solc filter as a polarized color filter and a color reproduction algorithm based on the optimization of the time-multiplexing matrix for the selected primary color filters. Our proof-of-concept prototype showed that our single SLM design can produce subtractive images with variable contrast and a wider color gamut than conventional LADs.

We propose HaptoFloater, a low-latency mid-air visuo-haptic augmented reality (VHAR) system that utilizes imperceptible color vibrations. When adding tactile stimuli to the visual information of a mid-air image, the user should not perceive the latency between the tactile and visual information. However, conventional tactile presentation methods for mid-air images, based on camera-detected fingertip positioning, introduce latency due to image processing and communication. To mitigate this latency, we use a color vibration technique; humans cannot perceive the vibration when the display alternates between two different color stimuli at a frequency of 25 Hz or higher. In our system, we embed this imperceptible color vibration into the mid-air image formed by a micromirror array plate, and a photodiode on the fingertip device directly detects this color vibration to provide tactile stimulation. Thus, our system allows for the tactile perception of multiple patterns on a mid-air image in 59.5 ms. In addition, we evaluate the visual-haptic delay tolerance on a mid-air display using our VHAR system and a tactile actuator with a single pattern and faster response time. The results of our user study indicate a visual-haptic delay tolerance of 110.6 ms, which is considerably larger than the latency associated with systems using multiple tactile patterns.

This paper presents a shadowless projection mapping system for interactive applications in which a target surface is frequently occluded from a projector with a user's body. We propose a delay-free optical solution for this critical problem. Specifically, as the primary technical contribution, we apply a large format retrotransmissive plate to project images onto the target surface from wide viewing angles. We also tackle technical issues unique to the proposed shadowless principle. First, the retrotransmissive optics inevitably suffer from stray light, which leads to significant contrast degradation of the projected result. We propose to block the stray light by covering the retrotransmissive plate with a spatial mask. Because the mask reduces not only the stray light but the achievable luminance of the projected result, we develop a computational algorithm that determines the shape of the mask to balance the image quality. Second, we propose a touch sensing technique by leveraging the optically bidirectional property of the retrotransmissive plate to support interaction between the user and the projected contents on the target object. We implement a proof-of-concept prototype and validate the above-mentioned techniques through experiments.

This article proposes HaptoMapping, a projection-based visuo-haptic augmented reality (VHAR) system, that can render visual and haptic content independently and present consistent visuo-haptic sensations on physical surfaces. HaptoMapping controls wearable haptic displays by embedded control signals that are imperceptible to the user in projected images using a pixel-level visible light communication technique. The prototype system is comprised of a high-speed projector and three types of haptic devices-finger worn, stylus, and arm mounted. The finger-worn and stylus devices present vibrotactile sensations to a user's fingertips. The arm-mounted device presents stroking sensations on a user's forearm using arrayed actuators with a synchronized hand projection mapping. We identified that the developed system's maximum latency of haptic from visual sensations was 93.4 ms. We conducted user studies on the latency perception of our VHAR system. The results revealed that the developed haptic devices can present haptic sensations without user-perceivable latencies, and the visual-haptic latency tolerance of our VHAR system was 100, 159, 500 ms for the finger-worn, stylus, and arm-mounted devices, respectively. Another user study with the arm-mounted device discovered that the visuo-haptic stroking system maintained both continuity and pleasantness when the spacing between each substrate was relatively sparse, such as 20 mm, and significantly improved both the continuity and pleasantness at 80 and 150 mm/s when compared to the haptic only stroking system. Lastly, we introduced four potential applications in daily scenes. Our system methodology allows for a wide range of VHAR application design without concern for latency and misalignment effects.

We perceive vibrotactile stimuli from smart devices such as smartphones when we use various applications. However, vibrators in these devices can present only a specific nearby resonant frequency with enough intensity to perceive, making it challenging to offer various vibrotactile stimuli. In this study, we propose a method to realize a vibrotactile presentation in a wide range of frequencies using airflow vibration generated by a built-in audio speaker of a smart device. We implemented a system based on the proposed method using a smartphone and experimented with measuring the airflow pressure. Moreover, we also propose the application of texture presentation using airflow.

Aiming at realizing novel vision augmentation experiences, this paper proposes the IlluminatedFocus technique, which spatially defocuses real-world appearances regardless of the distance from the user's eyes to observed real objects. With the proposed technique, a part of a real object in an image appears blurred, while the fine details of the other part at the same distance remain visible. We apply Electrically Focus-Tunable Lenses (ETL) as eyeglasses and a synchronized high-speed projector as illumination for a real scene. We periodically modulate the focal lengths of the glasses (focal sweep) at more than 60 Hz so that a wearer cannot perceive the modulation. A part of the scene to appear focused is illuminated by the projector when it is in focus of the user's eyes, while another part to appear blurred is illuminated when it is out of the focus. As the basis of our spatial focus control, we build mathematical models to predict the range of distance from the ETL within which real objects become blurred on the retina of a user. Based on the blur range, we discuss a design guideline for effective illumination timing and focal sweep range. We also model the apparent size of a real scene altered by the focal length modulation. This leads to an undesirable visible seam between focused and blurred areas. We solve this unique problem by gradually blending the two areas. Finally, we demonstrate the feasibility of our proposal by implementing various vision augmentation applications.

The projection of invisible data on visible images can facilitate seamless interactive projection, since data embedded in regular images is unobtrusive to human viewers. However, the previous techniques sacrificed one of the following key goals: 1) calibration-free setup; 2) full-color projection; or 3) high contrast image. In this paper, we propose a Pixel-level Infrared Light Communication (PILC) projector that achieves all these requirements by adding an infrared light source to the full-color projector. To provide a proof of concept, we built a functional prototype, evaluated its performance, and presented a basic application.

Recently, small and inexpensive portable projectors are commonly being used for presentations. For convenience, it is desired to perform a direct pointing operation on the screen without requiring to grip or mount any device and control the pointer intuitively. Therefore, this study proposes a new touch-operated large-screen projection system using acoustic vibration sensing and a projector-camera system. We apply a continuous signal in the inaudible range of an actuator to a surface and acquire its elastic compliance change as a resonance of sounds. We perform touch detection, position estimation, and pressure estimation by using machine learning techniques. Our actuator and sensor units are small and easy to install on a surface. Furthermore, our system can detect "true" touch without requiring any devices such as pens or pointers. We demonstrate a series of novel interactions for a large-screen projection system with our proposed technology through three applications.