In metaverse environments, we test whether visual harmony between avatar appearance and text design shapes trait impressions. Using a within-subject design that manipulates avatar hair color and text color/font, we find that congruent pairs increase satisfaction and affinity and convey traits more consistently, informing design guidelines for expressive communication.

In orthodontic treatment for children, a method has been recommended to strengthen bite force by chewing a tube in a certain rhythm for five minutes every day. However, ensuring that children keep the training daily is difficult due to repetitive, monotonous movements. Additionally, it is not easy for guardians to keep monitoring and ensure that children are doing the training properly, biting the tube with appropriate force. Therefore, we aim to develop an interactive serious game that utilizes bite force as an input device, allowing children to continue training properly and allowing guardians to monitor their training situation easily. This study introduces a simplified tube-based bite force measurement device to assess and record the force of the bites, as well as training games using the device.

Kemo-mimi means the dog- or cat-like ears on a humanoid character, or the ears of the animal itself. Kemo-mimi is often used as an element of the avatar’s appearance. It is generally considered that the posture of animal ears represents the animal’s emotional state. And the idea has been used as a technique for expressing emotions in many cartoon and animation works. But despite this fact, there are few examples of studies on the emotions that can be expressed by animal ears. Therefore, we decided to investigate the relationship between the posture of the animal ears and emotions and to establish a method of expressing emotions using the ears. In the experiments, three-dimensional animations of animal ears changing posture were presented to the subjects, and they were asked to answer the emotion corresponding to the posture. The results showed that there was a certain degree of a common understanding of people’s impressions concerning the animal ears. In this paper, we report the emotions that can be expressed by the posture of the animal ears as revealed in this study.

We introduce a novel making method for kinetic garments, which can change their shape and are thus used for decoration and self-expression. Such garments are generally realized by attaching devices to garments, and wiring and control systems are required to make them. In particular, wiring becomes complicated when many actuators are used. This issue is an obstacle for fashion designers and increases production difficulty. We propose a power supply and communication method for devices distributed on conductive fabric. This method allows the individual control of devices on the fabric without complicated wiring. We use soft hair-like bending actuators, called Smart Hair, as devices. The present study evaluates the application of the proposed method to Smart Hair.

Playing back vibrotactile signals through actuators is commonly used to simulate tactile feelings of virtual textured surfaces. However, there is often a small mismatch between the simulated tactile feelings and intended tactile feelings by tactile designers. Thus, a method of modulating the vibrotactile perception is required. We focus on fine roughness perception and we propose a method using a pseudo-haptic effect to modulate fine roughness perception of vibrotactile texture. Specifically, we visually modify the pointer's position on the screen slightly, which indicates the touch position on textured surfaces. We hypothesized that if users receive vibrational feedback watching the pointer visually oscillating back/forth and left/right, users would believe the vibrotactile surfaces more uneven. We also hypothesized that as the size of visual oscillation is getting larger, the amount of modification of roughness perception of vibrotactile surfaces would be larger. We conducted user studies to test the hypotheses. Results of first user study suggested that users felt vibrotactile texture with our method rougher than they did without our method at a high probability. Results of second user study suggested that users felt different roughness for vibrational texture in response to the size of visual oscillation. These results confirmed our hypotheses and they suggested that our method was effective. Also, the same effect could potentially be applied to the visual movement of virtual hands or fingertips when users are interacting with virtual surfaces using their hands.

Impaired articulation, chewing, or swallowing can have a strong impact on Quality of Life. These behaviors can be affected by decreased oral muscle function caused by aging and/or certain medical conditions. Aspiration pneumonitis is a major cause of death among elderly people, and is triggered by impaired swallowing, mainly due to decreased oral muscle strength. To reduce the risk of aspiration pneumonitis, elderly people are often advised to exercise and train their oral muscles. Unfortunately, such training tends to be monotonous and it is often difficult for trainees to detect the effects of the training, i.e., improved oral function. To address this issue, we developed an oral muscle training game called Squachu. Squachu is composed of a non-contact tongue-mouth-motion detection system and a game engine. As motion is tracked using a depth camera, players play the game via mouth and tongue motion and do not have to wear any specialized equipment. Thus, the game is hygienic and easily accessible for daily play.

The present study investigates the augmentation effect of airflow on the sensation of a virtual reality walk. The intensity of cutaneous sensation evoked by airflow during the real and virtual walk was measured. The airflow stimulus was added to the participant with passive vestibular motion and visual presentation. The result suggests that the sensation of walking was strongly increased by adding the airflow stimulus to the vestibular and optic presentations. The cutaneous sensation of airflow was perceived higher for the sitting participant than during a real walk in both a single and the combined stimuli. The equivalent speed of airflow for the sitting participant was lowered from the airflow speed in the real walk.

The Kitchen Knife Safety Educator (KKse) is a safety education system designed to teach children how to correctly use cooking knives. Cooking is important for children to learn about what they eat. In addition, that is also important for daily communication between children and their parents. However, it is dangerous for young children to handle cooking knives. Because of this danger, parents often try to keep their young children away from the kitchen. Our proposed system will contribute to not only improving children's cooking skills, but also improving communication between parents and children. The system composed of a virtual knife with haptic feedback function, a touch/force sensitive virtual food and a two-dimensional force sensitive cutting board. This system was developed to teach a fundamental cutting method, the “thrusting cut”. This paper describes the detail of the system.

This study investigated the implementation of a hand model and contact simulation method for the purpose of improving the reality of object manipulation in a virtual environment. The study focused both on the hand tracking method that takes advantage of nails and also contact simulation using a deformable hand model. The manipulation of an object using a hand, is known to make more frequent use of the fingertips and palm. The proposed method seeks hand form that minimizes the position and orientation errors on those areas. Deformation of the soft tissue of the hand is considered to have an effect on both visual reality and the physical state of contact. In our implementation, the deformation was simulated by FEM and the friction of contact was introduced by the penalty method. In addition, a model that is based on metaballs (or blobs) was employed to represent the smooth surface of the object and to eliminate the problem that derives from polygon modeling. Through experimental implementation, it was proved that object manipulation such as pinching and grasping are possible and that the update rate of simulation can be approximately 50 Hz.

This paper describes a passive stimulation of a body to evoke a walking sensation using a vestibular and haptic device while the real body of the user is sitting. It imparts a pseudo body image to the user through the real (physical) body of the user as a part of the virtual reality (VR) display system. The created walking sensation was evaluated by nine factors to analyze the complex nature of the walking sensation.

It is common to investigate the content of a box or bottle by shaking it. The authors have investigated virtual realization of the shaking interaction using a haptic device that can present inertial force. This paper reports experiments on the discrimination and estimation of content's weight through the shaking interaction. In an experiment, the characteristics of recognition by the subject were evaluated by comparing the real box with solid content and the virtual box with a simulation model. Further, evaluation with liquid content was conducted. The result proved that the device can present solid content in a similar accuracy to the real box. Moreover, it was suggested that the motion of the gravity center of the box has a significant effect on the recognition of the content weight.

We aim to develop a virtual-reality system that records a person's walking experience and gives other users the experience of his/her walking. We recorded stereo motion images of video cameras on a person's forehead with synchronous acceleration data of ankles. Then, we presented stereo motion images on a HMD with synchronous vibrations on soles of observer's feet. Observers reported better experience of vection, walking, and tele-existence from stereo images with vibrations than without vibrations. We recorded walking experiences of different body sizes including a child (130 cm tall) and those of a dog. Observers can partly experience child's walking and even dog's running.

The Five Senses Theatre project was established for the development of a basic technology that enables the user to relive a spatial motion of other persons as if the user him/her-self experienced it in person. This technology aims to duplicate a bodily experience performed in the real space and to pass it to the other person. In other words, the user experiences another person's body that moved in a real space as if the user moved in that real space. The spatial motion may be a walking tour to the world heritage, a legend run of a top athlete, and etc. More specifically, the system creates the sensation of a self-body motion without a voluntary motion of the user by providing physical motion to the real body. The sensation of self-body motion is generated by not only a visually induced vection but proprioceptive and tactile sensations of the body passively evoked. The Five Senses Theatre provides multisensory stimuli to the user to make use of the user's body as medium to project the valuable experience from others to the user's cognition. The research issues include the following: 1) Cognitive mechanism of passive stimulation perceived as an active motion sensation (pseudo agency), and body ownership transfer (virtual body). 2) Device development of the mutisensory display system, rendering/projection algorithms, and a lifelog data system.

The present paper describes the system to present a pseudo-walking sensation to a sitting participant. The vibration was added to the heel and toe to imitate cutaneous sensation of the sole during walking. The sound of footsteps was also provided to the participant through headphones simultaneously. In this sound presentation, we used a spatial sound of footsteps of another walker as well as own footstep sound. Another walker's sound was moved along several trajectories in a VR space. We conducted an experiment to elucidate the effect of third person's footstep sound moved differently on the walking sensation of a participant. The result showed that third person's sound enhanced not only walking sensation but also translational sensation of a sitting participant. Furthermore, the effect was the highest when third person's sound came from front to backward of the participant in a VR space.

This paper describes rendering characteristics of a virtual walk by a motion seat. The motion seat creates a small body motion in three dof (lift, roll, and pitch directions) to make the user feel as if the user him-/herself is walking despite sitting own body. We consider the actual self body is a medium to render the virtual body to share experiences of others by using the motion seat as part of the multisensory display. After setting the motion to optimally present a walking sensation, basic characteristics of perception levels of the virtual walk by the seat motion were measured and compared with those for a real walk. The result indicated that the perception levels of the virtual walk were around those of a real walk.

We aim to enable additional sensation when using an anthropomorphic finger motion interface. To do so, we applied a conventional method to generate pseduo-haptics. To control the amount of scroll resulting from finger displacement on a display, illusory resistance or pseudo friction was confirmed by subjective evaluation. We first clarified that this illusory resistance influences finger walking behavior such as stride or speed. An additional experiment conducted in public space verified this influence.

This paper describes an approach to interpolate the wind direction and velocity using two fans. Firstly, maps of the angle and velocity of the wind in relation to the speeds of the two fans are created. It was found that changes in the speeds of the two fans in the interpolation were not necessarily linear to the directional angles from the fans. Next, in the generation of wind, the speeds of the fans are inversely referenced from the maps using the target direction and velocity. An experiment confirmed that wind can be generated from an intermediate direction between the two fans. Through an evaluation using subjects, it was proven that a change in the direction of the wind could be recognized to some extent, although great differences in accuracy were observed among individuals.

We conducted a user study to clarify whether a tactile flow created by a matrix of vibrators in a seat pan simultaneously presented with an optical flow in peripheral vision enhances the perceived forward velocity of self-motion. A brief tactile motion stimulus consisted of four successive rows of vibration, and the inter-stimulus onset between the tactile rows was varied to change the velocity of the tactile motion. The experimental results show that the velocity of self-motion is significantly overestimated for rapid tactile flows and underestimated for slow tactile flows, compared with only optical flow or non-motion vibrotactile stimulation conditions.

We have developed a novel automultiscopic display, CAGRA (Computer-Aided Graphical Real-world Avatar), which can provide full parallax both horizontally and vertically without imposing any other additional equipment such as goggles on users. CAGRA adopts two axes of rotation so that it can distribute reflected light all around. In our previous work, it was proved that the display can present a black and white image of a 3D object at 1 Hz with both horizontal and vertical parallax. Here, we discuss two important problems that remain unsolved in our previous work: optimal parameter setting and the rendering method. As for finding optimal parameters, the relationship among the rotation speed of two axes and the diffusion angle of the holographic diffuser is discussed. The rendering method was implemented so that accurate view is presented in spite of the unique mechanical structure of CAGRA. Furthermore, the synchronization mechanism between projection of images and rotation of the mirror, which was also a remaining problem, is also implemented. Findings obtained through solving these problems clarify the characteristics of this novel display system, CAGRA, and lead to its further development.

This paper discusses the implementation of an automultiscopic omni-directional display. By rotating a flat-panel display whose viewing angle is strictly limited, or has sharp directivity, different and independent views are presented to different viewpoints. Design parameters including the resolution of viewpoints and update rate for viewers are clarified, and the effect of the refresh rate of display panels, speed of rotation, directivity of the panel on these parameters are discussed. Based on the discussion; the implementation of a prototype is reported. A directional filter using lenticular and slit array is devised, and its characteristic is evaluated. Also, a rendering algorithm that takes into account the effect of rotation of the display panel is proposed. Through experiments the feasibility of the proposed approach is confirmed.

In this paper, we describe a novel voice menu presentation method, the vCocktail, designed for efficient human-computer interaction in wearable computing. This method is devised to reduce the length of the serial presentation of voice menus by introducing spatiotemporal multiplexed voices with enhanced separation cues. Perception error in judging voice direction was measured first to determine appropriate directions in which and interval angles at which menu items were placed allowing the user a clear distinction among the multiple items. Then voice menu items were presented under spatiotemporally multiplexed conditions with several different settings of spatial localization, number of words, and onset interval. The results of the experiments showed that the subjects could hear items very accurately with localization cues and appropriate onset intervals. In addition, the proposed attenuating menu voice and crosstype spatial sequence of presentation increased the correct answer ratio effectively improving distinction between menu items. A correct answer ratio of 99.7 % was achieved in the case of four-item multiplexing when an attenuating voice and a 0.2 sec onset interval were used with the cross-type spatial sequence.

The electromyogram (EMG) signal, which is a kind of biomedical information, might be promising as a computer-human interface. The purpose in this paper is to make a brief report on implementation of the EMG interface in CAVE-clone display and propose the advantages of it. In general, it is expected that the environmental electromagnetic noise would disturb the novel EMG signals. We measured the EMG signals and obtained high signal-to-noise ratio in CABIN immersive multiscreen display constructed at the University of Tokyo. The observed signal-tonoise ratio would make EMG signals a reliable input channel with simple signal processing, and real time control of virtual objects was successfully performed. The advantages of the EMG signals will be proposed as a promising human interface in immersive multiscreen environments. The user would control virtual objects reflecting the activations of the user’s muscles to realize fine operations. Furthermore, by using this additional biosignal channel, the user might interact with virtual objects without definite motion.

Recently, there are various types of display systems that can present aural, visual and haptic information related to the user’s position. It is also important to present olfactory information related to the user’s position, and we focus on the spatiality of odor, which is one of its characteristics. In this research, we constructed and evaluated a wearable olfactory display to present the spatiality of odor in an outdoor environment. The prototype wearable olfactory display system treats odor in the gaseous state, and the odor air is conveyed to the user’s nose through tubes. Using this system, we also present the spatiality of odor by controlling the odor strength according to the positions of the user and the odor source. With this prototype system, the user can specify the position of the odor source in an outdoor environment. To improve this prototype system, we constructed another wearable olfactory display. Because odor is treated in the gaseous state, the first prototype system has some problems such as the large size of the device and unintentional leakage of the odor into the environment. To solve these issues, we developed and evaluated an advanced wearable olfactory display that uses an inkjet head device to treat odor in the liquid state.

Augmenting or annotating physical objects using steerable video projector and camera combinations works well in the range of the projection system. However, since these systems are usually mounted in fixed locations, users have to stay within a certain radius around that location. By using several of these AR-Projection systems, we can make the application roam these systems and thus follow the user beyond these limits. Using this method, we furthermore remove the need to hold the augmented objects in a suitable direction towards the projector and can reduce the possibility of shadowing the projection by the user. We describe an architecture based on our 'Projected Applications' which coordinates the systems used to project the augmentations and keeps the projected applications' states in sync between the projection systems.

This paper describes a wearable electronic mnemonics, the iFlashBack system that promotes human memorization process for the action the user performed in the real-world. The goal of the system is to exploit unlimited capacity of the human brain for storing the variety of information, by assisting memorization using recorded video and related information captured during interaction process to real-world objects. Memorization reinforcement is achieved by drawing attention, promoting rehearsal by flashing a video of interaction, and organizing memory by providing relevant information. The action of the user is captured by a cap-mounted miniature camera, an RFID reader, and arm posture sensors worn by the user. For video-aided rehearsal, a small HMD presents the captured scenes immediately after the user's interaction behavior and/or at appropriate times in the user's activity. We conducted basic experiments to demonstrate our idea. Two memory tasks - to change cup positions within grids and to memorize random dot images - were performed by subjects. The results suggested that the iFlashBack system could reinforce memorization of the user's action by motion-controlled video aided rehearsal.

In this paper, we propose a new way of augmenting our environment with information without making the user carry any devices. We propose the use of video projection to display the augmentation on the objects directly. We use a projector that can be rotated and in other ways controlled remotely by a computer, to follow objects carrying a marker. The main contribution of this paper is a system that keeps the augmentation displayed in the correct place while the object or the projector moves. We describe the hardware and software design of our system, the way certain functions such as following the marker or keeping it in focus are implemented and how to calibrate the multitude of parameters of all the subsystems.

This paper describes the methodology and prototype for an all-around display system for video avatar presentation in the real world. This system enables us to reconstruct a video avatar which users can look at from all around, by spinning a flat panel display which has a small viewing angle and changing the image on the display panel depending on the display's orientation.

We propose an approach to object manipulation. In the approach, the user's hand or fingers are represented by a set of interface points, the interaction force on each interface point is computed by the collision response computation algorithm, and the motion of the object is simulated based on the interaction force. We employed a fast collision response computation method to accommodate a large number of interaction points. Also, we devised a method for stable computation of object motion. We carried out experiments on interaction with models of varying complexity and demonstrated the feasibility of our approach for dexterous manipulations.

In this paper, we propose a new approach to building a virtual environment by using raw data from a real environment. To manage and analyze different types of datasets from many sensors and databases, we build and integrate different types of environments with different types of datasets, user interactions and rendering methods. To integrate the environments, we merge rendered images of each virtual environment according to the user's viewpoint, depth information, and reliability of rendered images, and display the resultant image to the user. In addition, we propose an integration environment by using an environment manager, which determines the user's viewpoint with respect to the integrated environments according to the user's input, and compose a display image from rendered images of the integrant environments. The reliability of the rendered images is determined by data accuracy and resolution, display resolution and rendering errors from rendering methodologies. By integrating four constructed environments according to the user's viewpoint, we can construct a photorealistic large-scale environment from different types of datasets. Furthermore, we implemented the prototype system in CABIN and demonstrated the scalability of this architecture.

We discuss application possibilities of virtual reality technologies such as immersive projection technology, to the field of genome science, and the guiding principles for the visualization of genome information are proposed. In addition, we develop a visualization methodology by using immersive projection technology for pair-wise comparison between cluster sets generated from different gene expression datasets. Our methodology could display the distribution of overlaps between two hierarchical cluster sets based on hepatocellular carcinomas and hepatoblastomas.

Describes the design and implementation of an artificial force display for immersive projection displays (IPD) such as CAVE or CABIN. In order to give the user maximum freedom of motion in the large virtual space generated by an IPD, it is necessary to use a portable force display that is self-contained on the user's body. Therefore, we developed a wearable force display called HapticGEAR. It is a backpack-type device that transmits applied forces to the wearer by using a wire-tension mechanism. The device is designed such that it minimizes fatigue for the wearer and does not interrupt the user's motion and view. Also, we integrated the device into CABIN and experimentally evaluated both the accuracy it achieved in tracing objects and the displayed haptic sensation.

In this paper we discuss application possibilities of virtual reality technology such as immersive projection technology to the field of genome science. The prototype of the visualization environment is implemented and used in analysis to elucidate the important genes in categorizing liver cancer.

Representation of soft and deformable objects in the virtual environment is an important topic for force feedback. To represent such deformable objects, there are two models: the spring-network model, and the FEM elastic model. For the haptic presentation of the sensation of softness, the problem of large deformation is not fatal, because softness is recognized through a relatively small deformation of the object during the interaction. Hence we apply the linear FEM model to represent a soft object. We propose an idea to avoid changing the displacement boundary condition by converting the displacement boundary condition caused by the user's finger into the equivalent force boundary condition. We have implemented and tested the method in a virtual environment with a force feedback device.

We propose a method of simulating and representing cutting force in real time. To reduce the computation time, we employed a simplified physical model for the simulation. We define the cutting edge as a finite set of discrete points (i.e., discrete edges). By calculating the force on each discrete edge, we obtain the approximate distribution of force on the edge.