Dexterous freehand manipulation in virtual reality offers rich interaction but is limited by physical reach. Existing indirect remote manipulation techniques often sacrifice this dexterity. We address this by defining “virtually direct” manipulation, a conceptual framework for techniques that break from a purely direct or indirect model by decoupling the virtual hand from the physical one. Within this framework, we present Hitchhiking Hands (HH), a novel implementation designed to preserve the rich dexterity of direct manipulation at a distance. HH allows users to instantly switch control between multiple pre-defined virtual hands using gaze. We evaluated HH in two user studies. A first study showed our approach, which consistently maintains direct-touch properties, surpasses an established baseline in 6DoF manipulation performance and embodiment. A second qualitative study revealed HH excels in structured spaces but is ill-suited for unstructured global tasks, highlighting a trade-off between flexibility and learnability.

The use of beyond-real interactions (BRI) expands the possibilities of virtual reality (VR) experiences by enabling novel capabilities, such as extending the range of manipulation or employing unusual body configurations. However, such interactions require users to adapt to novel sensorimotor mappings, which can lead to decreased performance and a poorer quality of user experience. Recently, proprioceptive suppression with noisy tendon electrical stimulation (n-TES) has been proposed to promote adaptation to BRI, though its effects have been reported to be limited. This study investigated the effects of proprioceptive suppression via n-TES on the adaptation process to BRI and the consequent changes in body schema, which have remained unexplored in prior research. Using a between-subjects design with 24 participants, we examined the effects of n-TES on a reaching task with Go-Go interaction, which extends virtual hands to enable users to reach farther than physically possible. Body schema changes were measured using a pointing task without visual feedback of the hand after adaptation to the Go-Go interaction. The results revealed that n-TES significantly altered the adaptation process by facilitating early learning and also enhanced body schema changes and aftereffects of adaptation under certain conditions. However, parameter fitting of the learning curves also suggested that n-TES might lead to a deterioration in the final level of performance achieved through adaptation. Our findings provide initial evidence that proprioceptive suppression by n-TES can contribute to facilitating sensorimotor adaptation within VR applications, while its practical utility may require adjusting stimulation to prevent potential side effects in the later stages of adaptation, specifically in trajectory smoothness.

Virtual reality (VR) enables beyond-real interactions (BRI) that transcend physical constraints, offering effective user experiences like extending a hand to grasp distant objects. However, adapting to novel mappings of BRI often reduces performance and the sense of embodiment. To address this, we propose using noisy tendon electrical stimulation (n-TES) to decrease proprioceptive precision. Previous studies have suggested that attenuating proprioceptive precision is crucial for sensory-motor adaptations. Thus, we hypothesize that n-TES, which has been shown to reduce proprioceptive precision and induce visual-dependent perception in VR, can enhance user adaptation to BRI. We conducted a user study using go-go interaction, a BRI technique for interacting with distant objects, to assess the effects of n-TES. Given the individual variability in n-TES response, participants first underwent a proprioceptive precision test to determine the optimal stimulation intensity to lower the proprioceptive precision from 5 levels $(\sigma=0.25-125\text{mA})$. Reaching tasks using a 2x2 within-participants design evaluated the effects of go-go interaction and n-TES on performance, subjective task load, and embodiment. Results from 24 participants showed that go-go interaction increased reaching time and task load while decreasing the sense of embodiment. Contrary to our hypothesis, n-TES did not significantly mitigate most of these negative effects of go-go interaction, except that perceived agency was higher with n-TES during go-go interaction. The limited effectiveness of n-TES may be due to participants' habituation or sensory adaptation during the tasks. Future research should consider the adaptation process to BRI and investigate different BRI scenarios.

Redirected walking (RDW) is a method for exploring virtual spaces larger than physical spaces while preserving a natural walking sensation. Expanding the range of visual manipulation gains that can be applied without causing discomfort is necessary to apply RDW in practice. Ankle tendon electrical stimulation (TES) can expand the range by inducing body tilt sensation and sway. Therefore, in this study, we proposed a locomotion method that applies ankle TES to RDW. In Experiment 1, we evaluated the effect of TES on the detection threshold (DT), which is the maximal gain at which visual manipulation remains unnoticed. The results indicated that the DT was expanded when TES was applied to induce the body tilt sensation in the same direction as the RDW's visual manipulation. Specifically, the pooled mean of the DT was expanded by more than 18%. In Experiment 2, we evaluated the applicability, a supplementary index for assessing locomotion techniques. The results demonstrated that ankle TES mitigates the reduction of the applicability, especially under a curvature gain of $\pm 0.3 [m^{-1}]$±0.3[m-1].

Hand redirection, which subtly adjusts a user's hand movements in a virtual environment, can modify perception and movement by providing real-time corrections to motor feedback. In the context of motor learning and rehabilitation, observing replays of movements has been shown to enhance motor function. The application of hand redirection to these replays by making movements appear larger or smaller than they actually are has the potential to improve motor function. However, the detection threshold for hand redirection, specifically in the context of motion replays, remains unclear, as it has primarily been studied in real-time feedback settings. This study aims to determine the threshold at which hand redirection during post-exercise replay sessions becomes detectable. We conducted two psychophysical experiments to evaluate how much discrepancy between replayed and actual movements can go unnoticed by users, both with hand redirection (N=20) and without (N=18). Our findings reveal a tendency for the amount of movement during replay to be underestimated. Furthermore, compared to conventional real-time hand redirection without replay, replay manipulations involving redirection applied during the preceding reaching task resulted in a significantly larger JND. These insights are crucial for leveraging hand redirection techniques in replay-based motor learning applications.

As a redirected walking technique, a method has been proposed to enable users to walk in an undulating virtual space even in a flat physical environment by setting the slope of the floor in the virtual environment to be different from that in the physical environment without causing discomfort. However, the slope range in which discrepancies between visual and proprioceptive sensations are not perceived is limited, restricting the slopes that can be presented. In this study, we proposed redirected walking using electrical stimulation of the Achilles and tibialis anterior muscle tendons, extending the applicable slope range of redirected walking without compromising the natural gait sensation. Electrical stimulation of the ankle tendons affects the proprioceptive sensation and gives the illusion of tilting in the standing posture, expanding the applicable slope range. Two experiments showed that the proposed method improved the experience of uphill and downhill walking in terms of the range of the virtual slope where a high naturalness of gait and a high congruency of visual and proprioceptive sensations are maintained. Notably, electrical stimulation of the Achilles tendons significantly improved the naturalness of the walking experience during virtual downhill walking, which has been considered more challenging in previous studies.

We present Virtual Reality Self Co-embodiment, a new method for post-stroke upper limb rehabilitation. It is inspired by mirror therapy, where the patient's healthy arm is involved in recovering the affected arm's motion. By tracking the user's head, wrists, and fingers' positions, our new approach allows the handicapped arm to control a digital avatar in order to pursue a reaching task. We apply the concept of virtual co-embodiment to use the information from the unaffected arm and complete the affected limb's impaired motion, which is our added unique feature. This requires users to mechanically involve the incapacitated area as much as possible, prioritizing actual movement rather than the sole imagination of it. As a result, subjects will see a seemingly normally functional virtual arm primarily controlled by their handicapped extremity, but with the constant support of their healthy limb's motion. Our experiment compares the task execution performance and embodiment perceived when interacting with both mirror therapy and our proposed technique. We found that our approach's provided sense of ownership is mildly impacted by users' motion planning response times, which mirror therapy does not exhibit. We also observed that mirror therapy's sense of ownership is moderately affected by the subject's proficiency while executing the assigned task, which our new method did not display. The results indicate that our proposed method provides similar embodiment and rehabilitation capabilities to those perceived from existing mirror therapy. This experiment was performed in healthy individuals to have an unbiased comparison of how mirror therapy's and VRSelfCo's task performance and degree of virtual embodiment compare, but future work explores the possibility of applying this new approach to actual post-stroke patients.

The time elapsed during a virtual reality (VR) experience is estimated to be short. Time estimation, a feeling related to timescales longer than a few seconds, is thought to be related to interoception. The shortening of time estimation may be caused by the VR experience distorting bodily awareness based on interoception; however, the details of this change have not been clarified. The characteristics of avatars are likely to affect bodily awareness. Therefore, based on the hypothesis that bodily awareness during a VR experience is altered by manipulating the avatar’s degree of anthropomorphism, this study investigated the effect of avatar anthropomorphism on bodily awareness and time estimation. In an experiment, participants performed a heartbeat discrimination task and a one-minute time estimation task, in the real world (Real condition), in the virtual world using a human avatar (VR-Human condition), and in the virtual world using a robot avatar (VR-Robot condition). The results showed that the perceived sense of agency was significantly lower in the VR-Human condition than in the VR-Robot condition, and interoceptive accuracy was significantly higher in the VR-Human condition than in the VR-Robot condition. However, there were no differences in time estimation between conditions. These results indicate that it is possible to conduct experiments to manipulate bodily awareness by manipulating the avatar characteristics. These findings can provide insights into understanding the relationship between bodily awareness and time estimation.

Hand redirection is a technique used in virtual reality (VR) to alter the virtual hand position from the real hand position, enabling extended interaction in VR. This technique capitalizes on the visual system’s dominance over proprioception, whereby discrepancies between the virtual hand mapping and the reality below a specific threshold are indiscernible to users. However, these detection thresholds are often minute. In this study, we explore the impact of haptic stimuli known as the ‘hanger reflex’ on the detection threshold of hand redirection, specifically concerning forearm pronation, which involves rotating the user’s palm downward. To achieve this, we conducted a user study that involved manipulating the degree of forearm rotation and measuring detection thresholds in three conditions: Neutral, where no force was applied; Inward, where haptic feedback was provided to match the direction of hand rotation through the hanger reflex; and Outward, where haptic feedback was provided in the opposite direction of hand rotation. The results demonstrate that, in the Neutral condition, the virtual degree of rotation can vary between 0.896 and 1.136 times the actual amount without being perceptible to users. However, under the Inward condition, this range increases to between 0.888 and 1.204 times the actual amount, representing a significant expansion of the detection threshold range. Consequently, our findings contribute to the field of upper limb rehabilitation.

This paper investigated the impact of a quantizer in analog-to-digital and digital-to-analog converters in communication devices on image quality when using deep learning-based joint source-channel coding modulation (JSCCM) for image transmission. In recent years, JSCCM, which efficiently encodes images and videos with low information entropy, has attracted great attention. JSCCM has a structure based on an autoencoder and determines the compression ratios for the image input by adjusting the number of IQ symbol output. The IQ symbol output from the encoder are allocated to symbol constellations with higher degrees of arbitrariness than those in typical square quadrature amplitude modulation and are therefore expected to be strongly affected by the quantization noise. In this paper, we employed quantization to the IQ symbol sequence and investigated its effect. Adjusting the quantizer's clipping ratio and the number of quantization bits, we examined the images' tolerance of the peak signal-to-noise ratio (PSNR). The simulation results showed that by adequately adjusting the clipping ratio, the image quality can be guaranteed to be equivalent to ideal conditions without quantization noise, and the number of required quantization bits that do not degrade the PSNR, was calculated.

To increase the flexibility of haptic feedback in virtual reality (VR), hand redirection (HR) has been proposed to shift the hand’s virtual position from its actual position. To expand the range of HR applications, a method to broaden the detection threshold (DT), which is the maximum amount of shift that can be applied without the user noticing, is required. Multisensory integration studies have revealed that the reliability of senses affects the weight of integration. To expand the DTs of HR, we propose a method to increase visual dominance in the integration of vision and proprioception by introducing noise to the latter, thereby decreasing its reliability through weak Gaussian white noise electrical stimulation ($\sigma$=0.5mA). The results of a user study comprising 22 participants (11 women and 11 men) confirm that noisy electrical stimulation significantly expands the DTs of HR with the mean range of DTs ($R_{DT}$) was 20.48° (SD =7.90) with electrical stimulation and 19.15° (SD =7.11) without electrical stimulation. Interestingly, this effect was only observed in women. The average $R_{DT}$ for men was 15.36° (SD =6.13) and 15.18°(SD=5.58), whereas that for women was 25. 61°(SD=5.89) and 23.12°(SD=6.21), with and without electrical stimulation, respectively. Electrical stimulation was mostly tolerable for the participants and did not affect embodiment or presence ratings. These results suggest that expansion of the DT without disturbing the user’s VR experience is feasible.

Deep joint source-channel coding (JSCC) has been attracting attention for achieving task-oriented communication. It replaces traditional information source coding and channel coding with a deep learning-based autoencoder, directly mapping information sources such as images to IQ symbols. For images, it is claimed to avoid the cliff effect and achieve a higher peak signal noise ratio (PSNR) even in low SNR regions. While related work has assumed various propagation channel models and validated the effectiveness of Deep JSCC, there are few reports confirming its principles through experiments. Specifically, to the best of our knowledge, there are no reported examples of experiments of Deep JSCC in 5G systems. In this paper, we present a proof-of-concept of Deep JSCC in a 5G system. We modified commercially available 5G base stations (gNB) and 5G terminals to enable input and output of IQ data from external devices. We connect the 5G devices using coaxial cables and attenuators, transmit and receive JSCC signals, and evaluate the PSNR. The results demonstrate that even when communicating at power levels lower than the minimum receiver sensitivity specified in the receiver’s datasheet, the image can be successfully restored with less than 1 dB degradation in PSNR compared with the simulation result.

Presenting haptic feedback in virtual reality (VR) is a long-standing challenge, with passive haptics being one way of presenting haptic feedback inexpensively. However, passive haptics requires props in physical environments that are co-located with their virtual counterparts, which is often not the case in the real world. Although redirected hands and other methods have previously been proposed to solve this problem, significant differences between the displayed and actual hand positions can cause the degradation of presence and sense of embodiment, limiting the range of presentable environments. In this study, we present a new hand displacement method called wormholes, in which the virtual hand is teleported discontinuously as the user inserts their hand into the hole. The experiment showed that the wormhole could maintain the sense of embodiment, presence, and task performance even with large hand displacements. Our method enables to apply passive haptics even when the actual and virtual environments are quite different, contributing to the realization of inexpensive and flexible haptic presentation in VR applications.

In this study, considering the characteristics of multisensory integration, we examined a method for improving redirected walking (RDW) by adding noise to the vestibular system to reduce the effects of vestibular inputs on self-motion perception. In RDW, the contradiction between vestibular inputs and visual sensations may make users notice the RDW manipulation, resulting in discomfort throughout the experience. Because humans integrate multisensory information by considering the reliability of each modality, by reducing the effects of vestibular inputs on self-motion perception, it is possible to suppress awareness of and discomfort during RDW manipulation and improve the effectiveness of the manipulation. Therefore, we hypothesized that adding noise to the vestibular inputs would reduce the reliability of the vestibular sensations and enhances the effectiveness of RDW by improving the relative reliability of vision. In this study, we used noisy galvanic vestibular stimulation (GVS) to reduce the reliability of vestibular inputs. GVS is a method of stimulating vestibular organs and nerves by applying small electrical currents to the bilateral mastoid. To reduce the reliability of vestibular inputs, we employed noisy GVS whose current pattern is white noise. We experimented with comparing the threshold of curvature gains between noisy GVS conditions and a control condition.

In this paper, we propose a novel redirected walking (RDW) technique that applies dynamic bending and curvature gains so that users perceive less discomfort than existing techniques that apply constant gains. Humans are less likely to notice continuous changes than those that are sudden. Therefore, instead of applying constant bending or curvature gains to users, we propose a dynamic method that continuously changes the gains. We conduct experiments to investigate the effect of dynamic gains in bending and curvature manipulation with regards to discomfort. The experimental results show that the proposed method significantly suppresses discomfort by up to 16 and 9% for bending and curvature manipulations, respectively.

In this paper, we present a study of redirected walking (RDW) that shifts the positional relationship between visual and auditory cues during curvature manipulation. It has been shown that, when presented with incongruent visual and auditory spatial cues during a localization task, human observers integrate that information based on each cue's relative reliability, which determines their final perception of the target object's location. This multi-modal integration model is known as maximum likelihood estimation (MLE). By altering the visual location of objects that users perceive in virtual reality (VR) through auditory cues during redirection manipulation, we expect fewer users to notice the manipulation, which helps increase the usable curvature gain. Most existing studies on MLE in multi-modal integration have used random-dot stereograms as visual cues under stable motion states. In the present study, we first investigated whether this model holds while walking in VR environment. Our results indicate that in a walking state, users' perceptions of the target object's location shift toward auditory cue as the reliability of vision decreases, in keeping with the trend shown in previous studies on MLE. Based on this result, we then investigated the detection threshold of curvature gains during redirection manipulation under a condition with congruent visual-auditory cues as well as a condition in which users' location perceptions of the target object are considered to be affected by the incongruent auditory cue. We found that the detection threshold of curvature gains was higher with incongruent visual-auditory cues than with congruent cues. These results show that incongruent multimodal cues in VR may have a promising application in the area of redirected walking.

Several redirected walking techniques have been introduced and analyzed in recent years, while the main focus was on manipulations in horizontal directions, in particular, by means of curvature, rotation, and translation gains. However, less research has been conducted on the manipulation of vertical movements and its possible use as a redirection technique. Actually, vertical movements are fundamentally important, e.g., for remotely steering a drone using a virtual reality headset.In this paper, we explored vertical gains, a novel redirection technique, which enables us to purposefully manipulate the mapping of the user’s physical vertical movements to movements in the virtual space and the remote space. This approach allows natural and more active physical control of a real drone. To demonstrate the usability of vertical gains, we implemented a telepresence drone and vertical redirection techniques for stretching and crouching actions using common VR devices. We conducted two user studies to investigate the effective manipulation ranges and its usability: one study using a virtual environment (VE), and one using a camera stream from a telepresence drone. The results revealed that our technique could manipulate a users vertical movement without her/his noticing.

Redirection makes it possible to walk around a vast virtual space in a limited real space while providing a natural walking sensation by applying a gain to the amount of movement in a real space. However, manipulating the walking path while keeping it and maintaining the naturalness of walking when turning at a corner cannot be achieved by the existing methods. To realize natural manipulation for turning at a corner, this study proposes novel “turning gains”, which refer to the increase in real and virtual turning degrees. The result of an experiment which aims to estimate the detection thresholds of turning gains indicated that when the turning radius is 0.5 m, discrimination is more difficult compared with the rotation gains (r = 0.0m).

In this study, we examined the effect of walking biomechanics, which occurs when the curvature of the walking path in virtual space is changed, while the actual walking path remains constant. Curvature gains and bending gains were used to change the virtual walking path. We found a significant difference in most biomechanical parameters when curvature manipulation and bending manipulation are applied compared with the case in which they are not applied. Some parameters were also suggested to depend on the visual sense or disagreement between the visual and other senses.

The contribution of this paper is to propose a novel Redirected Walking (RDW) technique that adopts manipulation gain in the roll direction. RDW is a technique that enables users to explore a large Virtual Environment (VE) while walking within a physically limited space by manipulating their virtual vision. Thus far, studies have determined the detection thresholds for translation, rotation, and curvature gains, but the thresholds are limited in the yaw direction. In contrast, in other research areas, movements in the yaw and roll directions have sometimes been addressed at the same time. In the present study, we investigated the detection threshold of inclination gain in the roll direction. We applied an inclination gain that was increased gradually while the participants walked 3 meters straight ahead. The results showed that users can detect an inclination gain of 1.93° on the left hand and 1.39° on the right hand.

A redirected walking (RDW) techniques enable users to walk around infinite virtual environments (VEs) in a finite physical space. In previous studies on RDW, many researchers have discussed manipulations in the yaw direction, but few have tackled with redirection in pitch and roll directions. We propose a novel VR system, which realizes pitch and roll redirections and allows users to experience walking on the 3D model of Mobius Strip in the VE.

This paper presents a novel interactive system that provides users with virtual reality (VR) experiences, wherein users feel as if they are ascending/descending stairs through passive haptic feedback. The passive haptic stimuli are provided by small bumps under the feet of users; these stimuli are provided to represent the edges of the stairs in the virtual environment. The visual stimuli of the stairs and shoes, provided by head-mounted displays, evoke a visuo-haptic interaction that modifies a user's perception of the floor shape. Our system enables users to experience all types of stairs, such as half-turn and spiral stairs, in a VR setting. We conducted a preliminary user study and two experiments to evaluate the proposed technique. The preliminary user study investigated the effectiveness of the basic idea associated with the proposed technique for the case of a user ascending stairs. The results demonstrated that the passive haptic feedback produced by the small bumps enhanced the user's feeling of presence and sense of ascending. We subsequently performed an experiment to investigate an improved viewpoint manipulation method and the interaction of the manipulation and haptics for both the ascending and descending cases. The experimental results demonstrated that the participants had a feeling of presence and felt a steep stair gradient under the condition of haptic feedback and viewpoint manipulation based on the characteristics of actual stair walking data. However, these results also indicated that the proposed system may not be as effective in providing a sense of descending stairs without an optimization of the haptic stimuli. We then redesigned the shape of the small bumps, and evaluated the design in a second experiment. The results indicated that the best shape to present haptic stimuli is a right triangle cross section in both the ascending and descending cases. Although it is necessary to install small protrusions in the determined direction, by using this optimized shape the users feeling of presence of the stairs and the sensation of walking up and down was enhanced.

Redirected walking allows users to explore a large virtual environment while there is a limitation of the room size. Previous works tried to present users straight path in a virtual environment while they walked on a curved path in reality. We expand a previous technique to present users a various curved path in a virtual environment while they walked on a particular curved path or a straight path with/without haptics. Furthermore, we propose a novel estimation methodology to quantify walking paths which user has thought he walked in reality. The data from our experiment shows that users feel walking a various curved path in VR as same as one-to-one mapping condition.