This study proposes a robotic massage system combining visual and tactile inspection for personalized therapy. Equipped with RGB-D cameras, a robot arm, and a force sensor, the system identifies massage areas based on body type via skeleton tracking. Using the kneading exploration method, muscle knots are localized efficiently, reducing location errors by up to 85.7% compared to conventional methods. The system also leverages VR visualization, showing massage actions through a robot arm and human-like therapist avatars. A study with ten participants found that human-like avatars, particularly a boy avatar, received higher ratings for pleasantness than a robot arm visualization, while pain perception showed no significant differences. These results highlight the influence of visual representation on psychological evaluations, enhancing the user experience. Future work will focus on improving interaction methods and integrating advanced physiological feedback to develop an adaptive Human-Robot Interaction system for physical and psychological care.

This study investigates how both the body part used to control a VR avatar and the avatar's appearance affect redirection detection thresholds. We conducted experiments comparing hand and foot manipulation of two types of avatars: a hand-shaped avatar and an abstract spherical avatar. Our results show that, irrespective of the body part used, the redirection detection threshold increased by 21% when using the hand avatar compared to the abstract avatar. Additionally, when the avatar's position was redirected toward the body midline, the detection threshold increased by 49% compared to redirection away from the midline. No significant differences in detection thresholds were observed between the hand and foot manipulations. These findings suggest that avatar appearance and redirection direction significantly influence user perception in VR environments, offering valuable insights for the design of full-body VR interactions and human augmentation systems.

Manipulating a non-humanoid body using a mapping approach that translates human body activity into different structural movements enables users to perform tasks that are difficult with their innate bodies. However, a key challenge is how to design an effective mapping to control non-analogous body parts with the human body. To address this challenge, we designed an articulated virtual arm and investigated the effect of mapping methods on a user’s manipulation experience. Specifically, we developed an unbranched 12-joint virtual arm with an octopus-like appearance. Using this arm, we conducted a user study to compare the effects of several mapping methods with different arrangements on task performance and subjective evaluations of embodiment and user preference. As a result, we identified three important factors in mapping: “Visual and Configurational Similarity”, “Kinematics Suitability for the User”, and “Correspondence with Everyday Actions.” Based on these findings, we discuss a mapping design for non-humanoid body manipulation.

Tactile interfaces, that can convey information to humans via tactile feedback, are still relatively rare. In this study we present a prototype ‘tactile scanner’, that fixes onto a user’s arm, and using arrays of capacitive sensors and vibratory motors, provides users with a sense of the proximity of objects near their arm. Through two experiments, we show that the device enables users to detect not just the position of objects, but also estimate their shapes and orientations. Finally, in a third experiment, we compare the user accuracy with the tactile scanner and their accuracy with real touch.

When humans generate stimuli voluntarily, they perceive the stimuli more weakly than those produced by others, which is called sensory attenuation (SA). SA has been investigated in various body parts, but it is unclear whether an extended body induces SA. This study investigated the SA of audio stimuli generated by an extended body. SA was assessed using a sound comparison task in a virtual environment. We prepared robotic arms as extended bodies, and the robotic arms were controlled by facial movements. To evaluate the SA of robotic arms, we conducted two experiments. Experiment 1 investigated the SA of the robotic arms under four conditions. The results showed that robotic arms manipulated by voluntary actions attenuated audio stimuli. Experiment 2 investigated the SA of the robotic arm and innate body under five conditions. The results indicated that the innate body and robotic arm induced SA, while there were differences in the sense of agency between the innate body and robotic arm. Analysis of the results indicated three findings regarding the SA of the extended body. First, controlling the robotic arm with voluntary actions in a virtual environment attenuates the audio stimuli. Second, there were differences in the sense of agency related to SA between extended and innate bodies. Third, the SA of the robotic arm was correlated with the sense of body ownership.

While previous studies have investigated virtual body embodiment in various forms, such as different colors, species, and transparency, little research has been conducted on elastic, non-rigid, or deforming bodies. If we can embody such elastic bodies, it will contribute to providing novel bodily experiences, such as being an octopus and developing a tele-operation system with deforming machines. We focused on skin deformation as the first step, and aimed to induce illusory experiences of having illusory sensation on deforming skin of a realistic virtual body using wearable tactile devices. The participants experience deforming skin illusions when a part of skin of the virtual arm is visually indented or stretched with synchronous tactile vibrations. The illusion is enhanced when the virtual arm’s location matches the real hand and when visual-tactile brush stroking is applied.

Perception of the ambient loudness affects feelings of interpersonal intimacy, while the voice loudness modulates the interpersonal distance. We aimed to test whether the interpersonal distance could be modulated by the ambient loudness as well as the loud speech emanating from a virtual human. Participants were asked to walk toward a virtual human with either the ambient sound or the virtual human’s speech. We found that the interpersonal distance to the virtual human increased only when the louder speech was emitted from the virtual human’s head irrespective of the understanding speech, while the walking time increased only when the ambient sound was louder, suggesting that the interpersonal distance is modulated by the virtual human’s voice but not by the ambient loudness.

Group exercise is more effective for gaining motivation than exercising alone, but it can be difficult to always find such partners. In this paper, we explore the experiences that joggers have with a virtual partner instead of a human partner and report on the results of two controlled experiments evaluating our approach. In Study 1, we investigated how participants felt and how their behav-ior changed when they jogged indoors with a human partner or with a virtual partner compared to solitary jogging. The virtual partner was represented either as a full-body, limb-only, or a point-light avatar displayed on smartglasses. In Study 2, we investigated the differences between the three representations as virtual partners for casual joggers in an outdoor setting. Based on our results, we propose implications for the design of virtual runners as casual jogging partners and speculate on their relationship with human users.

Massage therapy is crucial in the field of physical activities and sports to recover muscle fatigue and reduce stress. However, the physical workload on massage therapists is demanding, and the number of therapists is limited. To solve these problems, a massage robot that can move with human-like flexibility and delicacy was develop using a five-fingered robotic hand that mimicking a human hand. Moreover, the aim of this research is to develop a method of online estimation of a user's fatigue based on the muscle hardness in order to evaluate the effect of massaging from tactile information of the robot hand finger-tips to perform appropriate treatment. In this paper, we constructed a system about to measure muscle hardness based on reaction forces and measuring changes in muscle hardness and subjective fatigue over time before and after exercise. Then, the experiment of measuring muscle fatigue was conducted. As a result, the correlation coefficient between changes in muscle hardness and subjective fatigue highly correlated with five of the seven experimental participants. The result suggests the possibility that changes in the reaction force of muscles can be used as a measure for muscle fatigue recovery.

Humans have one own body. However, we can share a body with two different persons in a virtual environment. We have developed a shared body as an avatar that is controlled by two persons' actions. Movements of two subjects were continuously captured, and integrated into the avatar's motion with the ratios of 0:100,25:75, 50:50, 75:25, 100:0. They were not aware of integration ratios, and asked to reach cubes with the right hand. They felt body ownership and sense of agency to the shared avatar more when the responsible ratio was higher. The reaching path of the avatar's hand was shorter in the shared body condition (75:25) than the single body condition (100:0). These results suggest that we have some of body ownership and sense of agency to the shared body, and the task performance is improved by the shared body.

To capture a remote 3D image, conventional stereo cameras attached to a robot head have been commonly used. However, when the head and cameras rotate, the captured image in buffers is degraded by latency and motion blur, which may cause VR sickness. In the present study, we propose a method named TwinCam in which we use two 360° cameras spaced at the standard interpupillary distance and keep the direction of the lens constant in the world coordinate even when the camera bodies are rotated to reflect the orientation of the observer's head and the position of the eyes. We consider that this method can suppress the image buffer size to send to the observer because each camera captures the omnidirectional image without lens rotation. This paper introduces the mechanical design of our camera system and its potential for visual telepresence through three experiments. Experiment 1 confirmed the requirement of a stereoscopic rather than monoscopic camera for highly accurate depth perception, and Experiments 2 and 3 proved that our mechanical camera setup can reduce motion blur and VR sickness.

Humans can feel as if a fake body is their own body in the illusory body ownership. The illusion can be induced to a full body by a visual-tactile synchronicity or visual-motor synchronicity. In our previous study, illusory full body ownership of invisible body was elicited by synchronous movements of virtual gloves and socks. In this study, we aimed to investigate whether the spatial relationship is necessary for the full body illusion using a stimulus of scrambled body. In the scrambled body, positions of gloves and socks were scrambled. The results suggest that spatial relationship of body parts is necessary for the full body illusion.

With advancements in robotics, systems featuring wearable robotic arms teleoperated by a third party are appearing. An important aspect of these systems is the visual feedback provided to the third party operator. This can be achieved by placing a wearable camera on the robotic arm's “host,” or Main Body Operator (MBO), but such a setup makes the visual feedback dependant on the movements of the main body. Here we introduce a VR system called Parasitic Body to explore a VR shared body concept representative of the wearable robotic arms “host” (the MBO) and of the teleoperator (here called the Parasite Body Operator (PBO)). 2 users jointly operate a shared virtual body with a third arm: The MBO controls the main body and the PBO controls a third arm sticking out from the left shoulder of the main body. We focused here on the perspective dependency of the PBO (indeed, the PBO view is dependant of the movement of the MBO) in a “finding and reaching” task.

Pathogenic infection is usually invisible. Thus, we hardly care of preventing it in the daily life. We aimed to develop a virtual reality system to experience bidirectional infections using vision and tactile sensations; Users are infected by an avatar, and they do infect the avatar. When the avatar coughs toward a user, a cloud of visualized pathogen-droplets jet out and strike the user and objects on a table. At the same time, vibrations are presented to the user's chest. When the user coughs toward the avatar, the cough was detected by a throat microphone, and a cloud of visualized pathogen jets out. The avatar tries to avoid it unpleasantly, but infected. He is getting sick in bad face color. This system enables us to experience pathogenic infection with multimodal sensations in both perspectives (infected and infecting). Thus, it may contribute to increasing user's knowledge of infection and to facilitating prevention behaviors such as wearing a hygiene mask.

Illusory body ownership can be induced by visual-tactile stimulation or visual-motor synchronicity. We aimed to test whether a right thumb could be remapped to a virtual right arm and illusory body ownership of the virtual arm induced through synchronous movements of the right thumb and the virtual right arm. We presented the virtual right arm in synchronization with movements of a participant's right thumb on a head-mounted display (HMD). We found that the participants felt as though their right thumb became the right arm, and that the right arm belonged to their own body.

We present a novel assistive method for leading casual joggers by showing a virtual runner on see-through head-mounted display they worn. It moves at a constant pace specified in advance by them, and its motion synchronizes the user's one. People can always visually check the pace by looking at it as a personal pacemaker. They are also motivated to keep running by regarding it as a jogging companion. Moreover, proposed method overcomes safety problem of AR apps. Its most body parts are transparent so that it doesn't obstruct their view. This study, thus, may contribute to augment daily jogging experience.

The present study investigates the augmentation effect of airflow on the sensation of a virtual reality walk. The intensity of cutaneous sensation evoked by airflow during the real and virtual walk was measured. The airflow stimulus was added to the participant with passive vestibular motion and visual presentation. The result suggests that the sensation of walking was strongly increased by adding the airflow stimulus to the vestibular and optic presentations. The cutaneous sensation of airflow was perceived higher for the sitting participant than during a real walk in both a single and the combined stimuli. The equivalent speed of airflow for the sitting participant was lowered from the airflow speed in the real walk.

This paper describes a passive stimulation of a body to evoke a walking sensation using a vestibular and haptic device while the real body of the user is sitting. It imparts a pseudo body image to the user through the real (physical) body of the user as a part of the virtual reality (VR) display system. The created walking sensation was evaluated by nine factors to analyze the complex nature of the walking sensation.

We have developed a new type of robot which accompanies the healthcare professionals making medical rounds in patient's bedrooms at hospitals. This novel robot mainly executes two tasks: the carrying armamentarium of round medical supplies and the recording electronic health data on rounds. An omni-directional mobile mechanism and a human tracking control system to follow a specified medical professionals realize smooth transfer movement from the nurses' station to patient's bedroom. Electronic health data on rounds is automatically recorded by using a CCD camera and a voice recorder. When the robot is connected via a cable to a medical data server in nurses' station, the robot transmits the patients' data to the server and receives new patients' information.

We aim to develop a virtual-reality system that records a person's walking experience and gives other users the experience of his/her walking. We recorded stereo motion images of video cameras on a person's forehead with synchronous acceleration data of ankles. Then, we presented stereo motion images on a HMD with synchronous vibrations on soles of observer's feet. Observers reported better experience of vection, walking, and tele-existence from stereo images with vibrations than without vibrations. We recorded walking experiences of different body sizes including a child (130 cm tall) and those of a dog. Observers can partly experience child's walking and even dog's running.

We aim to develop a brain-machine interface (BMI) system that estimates user's gaze or attention on an object to pick it up in the real world. In Experiment 1 and 2 we measured steady-state visual evoked potential (SSVEP) using luminance and/or contrast modulated flickers of photographic scenes presented on a head-mounted display (HMD). We applied multiclass SVM to estimate gaze locations for every 2s time-window data, and obtained significantly good classifications of gaze locations with the leave-one-session-out cross validation. In Experiment 3 we measured SSVEP using luminance and contrast modulated flickers of real scenes that were online captured by a USB camera and presented on the HMD. We put AR markers on real objects and made their locations flickering on HMD. We obtained the best performance of gaze classification with highest luminance and contrast modulation (73-91% accuracy at chance level 33%), and significantly good classification with low (25% of the highest) luminance and contrast modulation (42-50% accuracy). These results suggest that the luminance-modulated flickers of real scenes through USB camera can be applied to BMI by using augmented reality technology.

The Five Senses Theatre project was established for the development of a basic technology that enables the user to relive a spatial motion of other persons as if the user him/her-self experienced it in person. This technology aims to duplicate a bodily experience performed in the real space and to pass it to the other person. In other words, the user experiences another person's body that moved in a real space as if the user moved in that real space. The spatial motion may be a walking tour to the world heritage, a legend run of a top athlete, and etc. More specifically, the system creates the sensation of a self-body motion without a voluntary motion of the user by providing physical motion to the real body. The sensation of self-body motion is generated by not only a visually induced vection but proprioceptive and tactile sensations of the body passively evoked. The Five Senses Theatre provides multisensory stimuli to the user to make use of the user's body as medium to project the valuable experience from others to the user's cognition. The research issues include the following: 1) Cognitive mechanism of passive stimulation perceived as an active motion sensation (pseudo agency), and body ownership transfer (virtual body). 2) Device development of the mutisensory display system, rendering/projection algorithms, and a lifelog data system.

This paper describes rendering characteristics of a virtual walk by a motion seat. The motion seat creates a small body motion in three dof (lift, roll, and pitch directions) to make the user feel as if the user him-/herself is walking despite sitting own body. We consider the actual self body is a medium to render the virtual body to share experiences of others by using the motion seat as part of the multisensory display. After setting the motion to optimally present a walking sensation, basic characteristics of perception levels of the virtual walk by the seat motion were measured and compared with those for a real walk. The result indicated that the perception levels of the virtual walk were around those of a real walk.

Like selective visual attention, the representation of multiple objects as an ensemble is a major strategy used by the visual system to overcome a bottleneck that the system cannot handle multiple objects simultaneously. Although simple image properties including mean size, orientation, and brightness as well as higher-level properties such as the average emotion and gender of faces, can be extracted from a group of items/images (see Alvarez, 2011 for review), it is not clear whether subjective values (e.g., attractiveness) are processed in a similar manner.

Human observers can utilize the light field or scene illumination to perceive objects [1]. To investigate effects of the light field on object perception or material perception, computer graphics is employed to render objects surfaces under a certain scene illumination. However, representations of various materials in computer graphics have some limitations even with BRDF (Bidirectional Reflectance Distribution Function) and BSSRDF (Bidirectional surface scattering reflectance distribution function). Thus, we aimed to develop a system for capturing and presenting various scene illuminations, and to investigate effects of scene illumination on real objects' surfaces.

Human observers perceive human and animal actions from 10-20 point lights attached to body joints. We aimed to investigate human perceptual sensitivity in the biomechanics of animals and humans. We created quadruped point-light stimuli based on motion-captured data of real dogs and of humans mimicking the actions of dogs. Human participants were asked to evaluate how natural the stimulus was as an action of a quadruped animal after observing randomized trials of dog actions and human mimicry. We found that participants generally rated dog actions as more natural than human mimicry. In a subsequent experiment, we presented inverted stimuli and found that the difference in ratings of authenticity between dog actions and human mimicry decreased. These results suggest that human observers can distinguish the biomechanics of animal actions from that of human mimicry, and that its sensitivity is particular to upright postures because of configural processing.

How is our perception sensitive to quadruped animals' action? We will present point-light displays of real quadruped animal actions h and human mimicry of those. Human observers can distinguish the real animal actions from that of human mimicry even with short time presentation (500ms), but the accuracy decreased with inverted (upside down) presentation. Thus, sensitivity of our perception is limited to upright postures of quadruped animals.

We demonstrate a virtual face of a robot with minimum humanity. The name of the robot is “shugo-robot”, which is after “shugo-rei (guardian spirit)“. The shugo-robot follows her/his master by using laser sensors, and takes greeting behaviors by gazing to master's friend and others. Thus, he/she is a communication agent on behalf of master. The head part of the robot has a camera, and finds human faces. Then, the robot's face gazes at a person nearest to the robot. This behavior makes users to believe the robot has an intention to communicate with them. The robot makes blinking at random timings to show the robot is alive, and several facial expressions to show her/his feelings.

Usually we steer a car using mainly visual information to perceive road's shape and bends. We developed a driving simulator with visual and/or tactile information guides to virtually present drivers' lateral position and to enhance their steering performance. The purpose of this study was to test effects of the cross-modal guide information on the driving performance. We found that the tactile guide improved driving accuracy more than the visual guide without any tradeoff of driving loads. Thus, the tactile information of virtual position of a car is useful for assisting and improving driver's performance with fewer loads.

The purpose of this study was to investigate ERD/ERS during hand movements and their imagery for developing ERD/ERS-utilized speed control on the brain-computer-interface driving simulator. We found clear ERD in contra-lateral cortex after 9-days learning of motor imagery. We developed a driving simulator equipped with ERD speed control, and showed a driver could control the speed of car.

We developed a driving simulator with visual and/or auditory information display to enhance the perception of lateral position of the driving car in real-time. The purpose of this study was to test effects of the cross-modal assistance information on the driving performance. We found the discrete visual assistance improved the driving accuracy, but increased the driving load. For auditory and audio-visual assistances, the continuous information improved the accuracy without increasing the load. Thus, the cross-modal information is useful for assisting and improving driver's performance with fewer loads.

One of well-known theories for motion sickness and VE (Virtual Environment) sickness is 'sensory conflict' theory. In this paper, we investigated whether the conflict between actual (extra-retinal) eye-movement and visually-simulated (retinal) eye-movement affects the VE-sickness. In results, we found that VE-sickness was significantly decreased by the control of observer's eye-movement with a stationary/moving fixation point. When the extra-retinal and retinal eye-movements were incongruent while the observer's head was actively moving, the VE-sickness was increased for sickness-sensitive observers. These results suggest that we can decrease VE-sickness by controlling eye-movements with a stationary/moving fixation point to remove conflict of extra-retinal and visual eye-movements. This is a new proposal of the way to decrease VE-sickness.