Users are often seated in the real environment, while their virtual avatars either remain standing stationary or move in virtual reality (VR). This creates posture inconsistencies between the real and virtual embodiment representations. The relationship between posture consistency in locomotion techniques and sense of presence in VR is still unclear. This study investigates how visual and somatosensory integration affects the sense of standing (SoSt) and the sense of self-motion (SoSm) when the sitting posture is varied slightly, including highlighting the importance of sitting posture for locomotion design in VR. The degree and occurrence of SoSt and SoSm were assessed by subjective experiments, and it was found that higher sitting and lower sitting postures present higher SoSt and lower SoSm, respectively. Invocation of SoSt also influences postural perception. Perception of travel distance varied according to the posture condition when identical visual flow was presented. The findings suggest that visual and somatosensory integration related to posture enhances SoSt and SoSm, and a sitting posture with a higher seating position is recommended in seated VR locomotion design.

Systems with occlusion capabilities, such as those used in vision augmentation, image processing, and optical see-through head-mounted display (OST-HMD), have gained popularity. Achieving precise (hard-edge) occlusion in these systems is challenging, often requiring complex optical designs and bulky volumes. On the other hand, utilizing a single transparent liquid crystal display (LCD) is a simple approach to create occlusion masks. However, the generated mask will appear defocused (soft-edge) resulting in insufficient blocking or occlusion leakage. In our work, we delve into the perception of soft-edge occlusion by the human visual system and present a preference-based optimal expansion method that minimizes perceived occlusion leakage. In a user study involving 20 participants, we made a noteworthy observation that the human eye perceives a sharper edge blur of the occlusion mask when individuals see through it and gaze at a far distance, in contrast to the camera system's observation. Moreover, our study revealed significant individual differences in the perception of soft-edge masks in human vision when focusing. These differences may lead to varying degrees of demand for mask size among individuals. Our evaluation demonstrates that our method successfully accounts for individual differences and achieves optimal masking effects at arbitrary distances and pupil sizes.

In underwater environments, the reflected ultrasonic waves from objects generally provide more than just information about their color and shape for object recognition. Previous studies have overlooked the influence of object pose on these wave components. It is crucial to investigate how these poses affect the reflected wave components because object poses can vary widely and are often unpredictable in real-world scenarios. In this work, we introduce a novel dataset comprising reflected wave components collected from objects made of various materials and observed from various angles. We also show the preliminary evaluations on the performance of machine learning-based material classification on object pose. Our results indicate that the accuracy is consistently high (≥ 91%) for known angles but significantly drops (< 60%) when dealing with unknown angles in most cases. Based on these evaluations, we suggest several directions for future research. Our dataset is available at https://github.com/Nyamotaro/U2R.

Hap’n’Roll is a wearable device that leverages the concept of a scroll to present, with a single motor, tactile sensations of various sizes, shapes, and textures. Hap’n’Roll is composed of two axes, a sheet, and one motor. By changing the number of sheet wraps, the thickness within the user’s hand can be adjusted. Additionally, using holes on the sheet to secure the fingertips, it can present a wide range of sizes and shapes. Unlike typical existing handheld shape-changing devices, Hap’n’Roll is not limited to cylindrical forms. Furthermore, by moving different materials attached on the sheet to the fingertips, it can also express different textures. A user study showed that Hap’n’Roll can convey at least three sizes (small, medium, and large) and four types of shapes (a cylinder, a rectangle, a cone, and a cup), with a shape and size identification accuracy of approx. 76.1%. The identification accuracy for shape alone was approx. 98.5%. Moreover, several applications were developed to showcase the effectiveness of Hap’n’Roll’s mechanism for various haptic feedback.

Various studies have suggested that altering the appearance of food can impact multisensory flavor perception. The cross-modal effect of such visual changes on gustation may allow for the presentation of food tastes that are difficult to express with simple combinations of taste stimuli. This cross-modal effect of visual changes on gustation holds potential for applications in gustatory displays. However, the current limitation of existing Head-Mounted Displays (HMDs) is their restricted vertical Field of View (FoV), which prohibits the display of images near the mouth while eating. This limitation may impede the cross-modal effect of visual changes on multisensory flavor perception. Additionally, the lack of visibility around the mouth area challenges the ease of eating. To address these issues, we design a Video See-Through (VST)-HMD with an expanded vertical FoV (approx. 100 [deg]). Using the HMD, we investigated how presenting visual information near the mouth affects the cross-modal effects of flavor perception and ease of eating. In our experiment, machine learning techniques were utilized to alter the appearance of food. However, the result showed no significant differences in the amount of cross-modal effects or the ease of eating between the groups with and without visual information near the mouth. As a discussion of this result, the participants may not direct their visual attention to the food when they put the food in their mouths. The experiment also examined whether visual changes alter the taste as well as the smell and texture of the food. The findings demonstrated that visual changes could present the smell and texture of the food following the modifications. This result was confirmed irrespective of the visibility near the mouth.

Although a variety of haptic devices are used for virtual reality (VR) and augmented reality (AR) experiences, few can present a wide range of softness-hardness of the surface of the virtual objects. We propose a haptic device that can present a wide range of softness-hardness by using a two-layered structure consisting of a pneumatic balloon and a mechanical piston. Through a series of user studies, we confirmed that the prototype can present five levels of softness and three levels of hardness, and that the prototype device improves the VR experience in terms of realism, enjoyment, and comfort for virtual objects with a variety of softness/hardness.

Virtual avatars are more and more often featured in Virtual Reality (VR) and Augmented Reality (AR) applications. When embodying a virtual avatar, one may desire to change of appearance over the course of the embodiment. However, switching suddenly from one appearance to another can break the continuity of the user experience and potentially impact the sense of embodiment (SoE), especially when the new appearance is very different. In this paper, we explore how applying smooth visual transitions at the moment of the change can help to maintain the SoE and benefit the general user experience. To address this, we implemented an AR system allowing users to embody a regular-shaped avatar that can be transformed into a muscular one through a visual effect. The avatar's transformation can be triggered either by the user through physical action (“active” transition), or automatically launched by the system (“passive” transition). We conducted a user study to evaluate the effects of these two types of transformations on the SoE by comparing them to control conditions where there was no visual feedback of the transformation. Our results show that changing the appearance of one's avatar with an active transition (with visual feedback), compared to a passive transition, helps to maintain the user's sense of agency, a component of the SoE. They also partially suggest that the Proteus effects experienced during the embodiment were enhanced by these transitions. Therefore, we conclude that visual effects controlled by the user when changing their avatar's appearance can benefit their experience by preserving the SoE and intensifying the Proteus effects.

The sense of reality is enhanced by presenting appropriate haptic feedback when the user interacts with a virtual object in virtual reality (VR). To present appropriate feedback, we often use a real object that resembles the virtual one to manipulate. However, such a real object is not always available. The user may feel a visuohaptic inconsistency between real and virtual objects when their shapes are different. To alleviate such an inconsistency, we propose a novel object synthesis method that combines the shape of the real object that the user manipulates in reality and the shape of the virtual object which was to be presented to the user in VR. In other words, this synthesized object, a chimera object, is created by transforming the part of the virtual object that the user would touch into a shape that is similar to the corresponding part of the real object while maintaining the other parts of the virtual object intact. The expected haptic sensation from the appearance of our chimera object is more consistent with the one produced by the real object. Therefore, the visuo-haptic inconsistency is expected to alleviate in VR, compared to the original virtual object. In addition, we propose an interactive system to support the design of a chimera object for users. To investigate the effectiveness of our method, we conducted two user studies. The first experiment confirmed that our proposed chimera object helps enhance visuo-haptic consistency. The second experiment confirmed that our system was effective for chimera object creation by users with acceptable system usability.

Common existing head-mounted displays (HMDs) for virtual reality (VR) provide users with a high presence and embodiment. However, the field of view (FoV) of a typical HMD for VR is about 90 to 110 [deg] in the diagonal direction and about 70 to 90 [deg] in the vertical direction, which is narrower than that of humans. Specifically, the downward FoV of conventional HMDs is too narrow to present the user avatar's body and feet. To address this problem, we have developed a novel HMD with a pair of additional display units to increase the downward FoV by approximately 60 ( 10+50) [deg]. We comprehensively investigated the effects of the increased downward FoV on the sense of immersion that includes presence, sense of self-location (SoSL), sense of agency (SoA), and sense of body ownership (SoBO) during VR experience and on patterns of head movements and cybersickness as its secondary effects. As a result, it was clarified that the HMD with an increased FoV improved presence and SoSL. Also, it was confirmed that the user could see the object below with a head movement pattern close to the real behavior, and did not suffer from cybersickness. Moreover, the effect of the increased downward FoV on SoBO and SoA was limited since it was easier to perceive the misalignment between the real and virtual bodies.

Augmented reality (AR) has been widely used that combines human with the digital world tightly at an unprecedented level, and various types of optical see-through head-mounted displays (OSTHMDs) have been actively developed to meet the requirement of everyday AR use. Correct mutual occlusion between real and virtual objects is often necessary for displaying realistic virtual images for users in the AR application scenarios. Some optical designs have been proposed to realize mutual occlusion by means of an OSTHMD in recent years. However, all of them support a limited field of view (FOV) that is much narrower than that of a natural human. The main limiting factor of the FOV of general OSTHMDs is the limited numerical aperture (NA) of the lenses. To address the problem, we propose an OSTHMD based on the double ellipsoidal mirror structure to avoid a stack of lenses and to achieve a wide FOV close to that of the naked eye with small distortion. A pair of imaging lenses are carefully arranged, then assembled between the two ellipsoidal mirrors with a pinhole mask to improve image quality. An experiment using a monocular prototype shows that a sharp see through view is achieved which remains clear regardless of the focus of the eye-simulating camera. An image undistortion algorithm is also developed to obtain a full-view display, enabling a virtual image to be displayed spanning a super wide FOV of H160°×V74°. Finally, per-pixel mutual occlusion of a wide FOV of H122°×V74° is realized by placing a spatial light modulator (SLM) in front of the entrance pupil.

In recent years, human augmentation has attracted much attention. One type of human augmentation, motion augmentation makes perceived motion larger than in reality, and it can be used for a variety of applications such as rehabilitation of motor functions of stroke patients and a more realistic experience in virtual reality (VR) such as redirected walking (RDW). However, as augmented motion becomes larger than the real motion, a variety of senses that accompany will be more inconsistent with those perceived from somatic sensations, which will cause a severe sense of discomfort. To address the problem, we focus on kinesthetic illusions that are psychological phenomena where a person feels as if his or her own body is moving. Kinesthetic illusions are expected to fill the gap between the intended augmented motion and perceived physical motion. However, it has not been explored if and how large kinesthetic illusions are produced while a user is moving their limbs voluntarily in VR. To expand the knowledge on kinesthetic illusions, we have conducted two user studies on the impact of tendon vibration and visual stimuli on kinesthetic illusions. First experiment confirmed that the perceived elbow angle becomes larger than the actual angle when presented with tendon vibration. Second experiment revealed that the increase of the perceived elbow angle was about 20 degrees when both tendon vibration and visual stimuli were presented whereas it was about 10 degrees when only visual stimuli were presented. Through these experiments, it has been confirmed that combining tendon vibration and visual stimulation enhances kinesthetic illusions.

In wearable computing environments, users acquire visual information in various scenes using a head mounted display (HMD). However, this induces problems due to their differences from conventional displays such as smartphone and e-books. In this research, we focused on the problem of vertical shock caused by walking. This problem interferes with seeing information on HMDs. In this paper, we discuss selection of font shapes to minimize the effect of this problem. If we can clarify the characteristics of the readability of fonts in wearable computing environments, the application designers can select fonts that strike a balance between intended design elements and readability. In this paper, we first investigated the characteristics of fonts used in Japan considering the HMD swing that results from walking, from the viewpoints of readability (text readability) and legibility (ease of letter recognition) using six different fonts. From this evaluation, we find that fonts with very thin horizontal lines and with very thin horizontal and vertical lines should not be presented on HMDs.

In recent years, augmented reality (AR) technology has received attention. AR technology is expected to be widely used in various day-to-day activities. Many researchers have proposed methods that use AR technology for various purposes. However, there is a possibility that displayed AR objects cause anxiety or spoil scenic views for the user, as the objects can hinder the view and the appearances can change to several forms. Therefore, it is important to consider how we should display the AR objects depending on its purpose. In this study, we have investigated how participants behave when they are shown different AR objects to lead them to the left/right side on an outdoor road. As a result, we confirmed that we should use the AR object whose appearance blends with the surroundings when the object is a sign to lead people on outdoor.

A human gaze not only has the meaning that someone is looking at something, but also has various effects on the surrounding people. For example, previous studies have shown that attention is induced in the gaze direction of others and that consciousness of being seen by others affects human behavior. In the future, android robots will play an active role in society. We can predict that communication and information transmission with such gaze information will be possible between humans and robots. However, to the best of our knowledge, no study has yet investigated the effect of robot gazes on people in a situation where many robots exist ubiquitously in daily life. Therefore, in this study, we aim to evaluate the attention inducing effect of the gaze of the face robots installed in the daily living environment. In this paper, we present several examples of services in a situation, where face robots are established ubiquitously in daily life. We verified the attention inducing effect in three test cases. As a result of the experiment, the number of times when the participants focused their attention in the direction of the gaze of the face robot was small. However, the feasibility of the service was demonstrated through the results of the experiment, such as surrounding situation, user's consciousness about the face robots, and attention induction timing.

In recent years, many studies relating biometric information and feelings have been done. Among them, in this paper, we aim to verify the influence on ourselves and surroundings by visualizing emotions. Have a plurality of subjects perform a simple task with a pulse wave sensor attached to the ear. Visualize one's emotion by color and observe how the change of the pulse waves of the subjects who saw it. Finally, take a questionnaire and verify whether the pulse wave matches the actually feeling. Also, although the emotion to be visualized should be displayed in real time in the actual experiment, it was assumed that the experimenter intentionally displayed the color in this experiment. In this case, it is also possible to evaluate the change in the pulse wave of the subject when the emotion different from the actual emotion is displayed.

This paper describes a participatory theatrical performance named YOUPLAY. In YOUPLAY, a general person who is usually treated as an audience member becomes an actor and plays a character role in the performance. He/she wears several types of equipments such as sensors and a microphone, and experiences an interactive story. We found during the development and actual use of the system that such interactive systems need robust activity recognition techniques. We explain the details of the system for YOUPLAY and present the knowledge gathered from 80 actual stages.

With the recent fall in the prices of small and high resolution projectors, it has become increasingly popular to project images onto a screen.

Recently, environments in which users can always see information is becoming realistic due to the development of wearable computing. Researchers have proposed various information presentation methods for different situations. However, in wearable computing environments, since the user does not always need a particular piece of information, there should a specified duration for which the system can show information that does not have any specific importance for the user. In this research, we propose a system that prompts the user to become unconsciously attracted to certain information by presenting visual information related to their interest. Our system utilizes the concept of the priming effect, whereby individuals are affected by content that has been presented before. Evaluation results confirmed that presenting information prompts the user to obtain the intended information. We also implemented a prototype of the system, Primer Streamer, that presents information related to user interest.

Various wearable computing devices face problems with their power supplies, communication channels, and placement. Conductive clothes can resolve these problems, but it is still difficult to know the positions of devices on the conductive fabric. Therefore, we have devised a method to detect the positions of such devices by using a camera. To detect the positions our method blinks the LEDs on the devices according to their ID. Additionally, we propose several methods to shorten the time for detection. An experimental evaluation confirmed that compared with conventional method our methods reduce the time to detect the positions of the devices.