Virtual reality, which enables users to engage in physical activities in ways distinct from those in the real world, is increasingly recognized for its potential to enhance motor skill acquisition. Research on co-embodiment learning, in which instructors and learners utilize a single avatar that represents a weighted average of their movements, has demonstrated its efficacy in facilitating motor skill development. However, the current implementation of co-embodiment learning necessitates the real-time participation of instructors proficient in both virtual reality and co-embodiment, which poses challenges for its widespread adoption. To address this limitation, this study proposed a method for developing instructors trained on human motor data to effectively support motor skill learning through co-embodiment. The AI model was trained using supervised learning on data obtained from human motor learning sessions that employed co-embodiment. To evaluate the performance of the AI instructor, we compared the learning performance in co-embodiment learning with that of the AI instructor, recorded human instructor data, and a human instructor as well as in solo learning. The results showed that practicing with the AI instructor significantly improved learning efficiency compared with practicing alone or with recorded data and was comparable to that achieved by practicing with a human instructor.

Emotional contagion is a phenomenon in which one’s emotions are transmitted among individuals unconsciously by observing others’ emotional expressions. In this paper, we propose a method for mediating people’s emotions by triggering emotional contagion through artificial bodily changes such as pseudo tears. We focused on shedding tears because of the link to several emotions besides sadness. In addition, it is expected that shedding tears would induce emotional contagion because it is observable by others. We designed an eyeglasses-style wearable device, Teardrop glasses, that release water drops near the wearer’s eyes. The drops flow down the cheeks and emulate real tears. The study revealed that artificial crying with pseudo tears increased sadness among both wearers and those observing them. Moreover, artificial crying attenuated happiness and positive feelings in observers. Our findings show that actual bodily changes are not necessary for inducing emotional contagion as artificial bodily changes are also sufficient.

Humans can estimate the shape of a wielded object through the illusory feeling of the mass properties of the object obtained using their hands. Even though the shape of hand-held objects influences immersion and realism in virtual reality (VR), it is difficult to design VR controllers for rendering desired shapes according to the perceptions derived from the illusory effects of mass properties and shape perception. We propose Transcalibur, which is a hand-held VR controller that can render a 2D shape by changing its mass properties on a 2D planar area. We built a computational perception model using a data-driven approach from the collected data pairs of mass properties and perceived shapes. This enables Transcalibur to easily and effectively provide convincing shape perception based on complex illusory effects. Our user study showed that the system succeeded in providing the perception of various desired shapes in a virtual environment.

Redirection makes it possible to walk around a vast virtual space in a limited real space while providing a natural walking sensation by applying a gain to the amount of movement in a real space. However, manipulating the walking path while keeping it and maintaining the naturalness of walking when turning at a corner cannot be achieved by the existing methods. To realize natural manipulation for turning at a corner, this study proposes novel “turning gains”, which refer to the increase in real and virtual turning degrees. The result of an experiment which aims to estimate the detection thresholds of turning gains indicated that when the turning radius is 0.5 m, discrimination is more difficult compared with the rotation gains (r = 0.0m).

During group decision-making, a group with an imbalanced minority and majority typically suffers normative bias. This bias creates socio-emotional conflict and decreases group member consent and decision power. Thus, we propose a novel video-chat system, "Divided Presence," which aims to reduce this bias by equalizing the apparent number of majority and minority participants using a pseudo-population increase. Members of a discussion use computer graphics avatars on monitors instead of their actual video appearances during communication. The apparent number of discussion members increases when two avatars are assigned to an arbitrary minority member; the avatars then speak on behalf of him or her. We evaluate the system with a three-person consensus game, where a minority member is assigned two avatars. The results show that our system increases the degree of consent among the majority participants when they finally agree to the minority opinion.

In this study, we examined the effect of walking biomechanics, which occurs when the curvature of the walking path in virtual space is changed, while the actual walking path remains constant. Curvature gains and bending gains were used to change the virtual walking path. We found a significant difference in most biomechanical parameters when curvature manipulation and bending manipulation are applied compared with the case in which they are not applied. Some parameters were also suggested to depend on the visual sense or disagreement between the visual and other senses.

The contribution of this paper is to propose a novel Redirected Walking (RDW) technique that adopts manipulation gain in the roll direction. RDW is a technique that enables users to explore a large Virtual Environment (VE) while walking within a physically limited space by manipulating their virtual vision. Thus far, studies have determined the detection thresholds for translation, rotation, and curvature gains, but the thresholds are limited in the yaw direction. In contrast, in other research areas, movements in the yaw and roll directions have sometimes been addressed at the same time. In the present study, we investigated the detection threshold of inclination gain in the roll direction. We applied an inclination gain that was increased gradually while the participants walked 3 meters straight ahead. The results showed that users can detect an inclination gain of 1.93° on the left hand and 1.39° on the right hand.

A redirected walking (RDW) techniques enable users to walk around infinite virtual environments (VEs) in a finite physical space. In previous studies on RDW, many researchers have discussed manipulations in the yaw direction, but few have tackled with redirection in pitch and roll directions. We propose a novel VR system, which realizes pitch and roll redirections and allows users to experience walking on the 3D model of Mobius Strip in the VE.

This paper presents a novel interactive system that provides users with virtual reality (VR) experiences, wherein users feel as if they are ascending/descending stairs through passive haptic feedback. The passive haptic stimuli are provided by small bumps under the feet of users; these stimuli are provided to represent the edges of the stairs in the virtual environment. The visual stimuli of the stairs and shoes, provided by head-mounted displays, evoke a visuo-haptic interaction that modifies a user's perception of the floor shape. Our system enables users to experience all types of stairs, such as half-turn and spiral stairs, in a VR setting. We conducted a preliminary user study and two experiments to evaluate the proposed technique. The preliminary user study investigated the effectiveness of the basic idea associated with the proposed technique for the case of a user ascending stairs. The results demonstrated that the passive haptic feedback produced by the small bumps enhanced the user's feeling of presence and sense of ascending. We subsequently performed an experiment to investigate an improved viewpoint manipulation method and the interaction of the manipulation and haptics for both the ascending and descending cases. The experimental results demonstrated that the participants had a feeling of presence and felt a steep stair gradient under the condition of haptic feedback and viewpoint manipulation based on the characteristics of actual stair walking data. However, these results also indicated that the proposed system may not be as effective in providing a sense of descending stairs without an optimization of the haptic stimuli. We then redesigned the shape of the small bumps, and evaluated the design in a second experiment. The results indicated that the best shape to present haptic stimuli is a right triangle cross section in both the ascending and descending cases. Although it is necessary to install small protrusions in the determined direction, by using this optimized shape the users feeling of presence of the stairs and the sensation of walking up and down was enhanced.

"Mirroring" refers to the unconscious mimicry of another person's behaviors, such as their facial expressions. Mirroring has many positive effects, such as enhancing closeness and improving the flow of a conversation, which enriches the quality of communication. Our study set out to devise a means of evoking these positive effects in a video chat without any conscious effort of participants. We constructed a videophone system, called FaceShare, which can deform the user's face into a smile in response to their partner's smiling. That is, our system generates mirroring by producing a pseudo-smile through image processing. We conducted an experiment in which pairs of participants had brief conversations via FaceShare. The results implied that mirroring using the pseudo-smile lets the mimicker, whose face is deformed according to the expressions of their partner, feel a closeness, and improves the flow of the conversation for both the mimicker and the mimickee, who sees the mimicker's deformed face.

As intelligent agents learn to behave increasingly autonomously and simulate a high level of intelligence, human interaction with them will be increasingly unpredictable. Would you accept an unexpected and sometimes irrational but actually correct recommendation by an agent you trust? We performed two experiments in which participants played a game. In this game, the participants chose a path by referring to a recommendation from the agent in one of two experimental conditions:the correct or the faulty condition. After interactions with the agent, the participants received an unexpected recommendation by the agent. The results showed that, while the trust measured by a questionnaire in the correct condition was higher than that in the faulty condition, there was no significant difference in the number of people who accepted the recommendation. Furthermore, the trust in the agent made decision time significantly longer when the recommendation was not rational.

To effectively utilize gaze in one-to-many communication, robots must direct and avert their gaze with appropriate timing. However, few studies have focused on this aspect of communication because of the difficulty in detecting the position of many gaze recipients and directing appropriate gazes to each. This paper proposes a system of robotic eyes that can send averted or direct gazes simultaneously to multiple recipients by switching between convex and hollow eye shapes, which we call Transgazer.

Redirected walking allows users to explore a large virtual environment while there is a limitation of the room size. Previous works tried to present users straight path in a virtual environment while they walked on a curved path in reality. We expand a previous technique to present users a various curved path in a virtual environment while they walked on a particular curved path or a straight path with/without haptics. Furthermore, we propose a novel estimation methodology to quantify walking paths which user has thought he walked in reality. The data from our experiment shows that users feel walking a various curved path in VR as same as one-to-one mapping condition.

Humans are capable of haptically perceiving the shape of an object by simply wielding it, even without seeing it. On the other hand, typical hand-held controllers for virtual reality (VR) applications are pre-designed for general applications, and thus not capable of providing appropriate haptic shape perception when wielding specific virtual objects. Contradiction between haptic and visual shape perception causes a lack of immersion and leads to inappropriate object handling in VR. To solve this problem, we propose a novel method for designing hand-held VR controllers which illusorily represent haptic equivalent of visual shape in VR. In ecological psychology, it has been suggested that the perceived shape can be modeled using the limited mass properties of wielded objects. Based on this suggestion, we built a shape perception model using a data-driven approach; we aggregated data of perceived shapes against various hand-held VR controllers with different mass properties, and derived the model using regression techniques. We implemented a design system which enables automatic design of hand-held VR controllers whose actual shapes are smaller than target shapes while maintaining their haptic shape perception. We verified that controllers designed with our system can present aimed shape perception irrespective of their actual shapes.

Agents like human, such as humanoid robots or avatars can be felt as if they have and communicate and communicate due to manipulation of the bodily information. Meanwhile, as in the case of Internet bot, it is still difficult to communiate the emotion described as text, let alone empathizing due to degradation of information online. The current study proposes a method for experiencing emotion on the Internet by reproducing a mechanism of evoking emotion. This method evokes a number of emotions described on the Web, by changing of self-physiological perception with sensory stimuli. To investigate the feasibility of our method, we made a system named "Communious Mouse." This system rewrites the perception of self-skin temperature and pulse in a palm by presenting vibration and thermal stimulation through a mouse device for evoking emotion. The current paper discusses the feasibility of our method based on the obtained feedbacks through an exhibition of the system.

We propose a navigation interface, which enables users to control locomotion and the speed intuitively in virtual environments constructed with spherical images. In these virtual environments, users can only pass through lines where the camera has passed through during recording. Therefore, it is necessary to inform users of the accessible directions. Furthermore, velocity control is needed for close exploration in virtual environments. Therefore, we propose the Motive Compass input interface, which intuitively shows users the accessible directions and enables them to control the velocity of locomotion. We conducted a large-scale demonstration experiment in a real exhibition and showed that our interface more effectively presents accessible directions and enables users to control their velocities.

Lately, mobile augmented reality (AR) has become very popular and is used for many commercial and product promotional activities. However, in almost all mobile AR applications, the user only views annotated information or the preset motion of the virtual object in an AR environment and is unable to interact with the virtual objects as if he/she were interacting with real objects in the real environment. In this paper, in an attempt to realize enhanced intuitive and realistic object manipulation in the mobile AR environment, we propose an integrated view-input AR interaction method, which integrates user device manipulation and virtual object manipulation. The method enables the user to hold a 3D virtual object by touching the displayed object on the 2D touch screen of a mobile device, and to move and rotate the object by moving and rotating the mobile device while viewing the held object by way of the 2D screen of the mobile device. Based on this concept, we implemented three types of integrated methods, namely the Rod, Center, and Touch methods, and conducted a user study to investigate the baseline performance metrics of the proposed method on an AR object manipulation task. The Rod method achieved the highest success rate (91%). Participants' feedback indicated that this is because the Rod method is the most natural, and evoked a fixed mental model that is conceivable in the real environment. These results indicated that visualizing the manipulation point on the screen and restricting the user's interactivity with virtual objects from the user's position of view based on a conceivable mental model would be able to aid the user to achieve precise manipulation.

Surface and object recognition is of significant importance in ubiquitous and wearable computing. While various techniques exist to infer context from material properties and appearance, they are typically neither designed for real-time applications nor for optically complex surfaces that may be specular, textureless, and even transparent. These materials are, however, becoming increasingly relevant in HCI for transparent displays, interactive surfaces, and ubiquitous computing. We present SpecTrans, a new sensing technology for surface classification of exotic materials, such as glass, transparent plastic, and metal. The proposed technique extracts optical features by employing laser and multi-directional, multi-spectral LED illumination that leverages the material's optical properties. The sensor hardware is small in size, and the proposed classification method requires significantly lower computational cost than conventional image-based methods, which use texture features or reflectance analysis, thereby providing real-time performance for ubiquitous computing. Our evaluation of the sensing technique for nine different transparent materials, including air, shows a promising recognition rate of 99.0%. We demonstrate a variety of possible applications using SpecTrans' capabilities.

This paper proposes a high-definition digital display case for manipulating a virtual exhibit that has linking mechanisms. This technique enhances the understanding of dynamic exhibits. It is difficult to construct interactive contents of dynamic virtual exhibits, because measuring the mechanism invokes the risk of an exhibit's deterioration, and it takes tremendous efforts to create a fine spun computer graphics (CG) model for mechanisms. Therefore, we propose an image-based interaction method that uses image-based rendering to construct interactive contents for dynamic virtual exhibits using the interpolation between exhibit pictures with a number of deformational conditions and viewpoints. Using this method, we construct a high-definition digital showcase and exhibit the interactive content at a museum to evaluate the availability of our system.

In this paper we developed the free-viewpoint curved surface shape display, using the effect of visuo-haptic interaction. In our research, we aim to realize the simple mechanic visuo-haptic system with which we can touch various objects with our real hands. We proposed the free-viewpoint shape display system in which users can touch a virtual object with various shape through the tablet device, while walking around the object. The system figured the difference of shape between the real object and the virtual one from the current viewpoint, and calculates the amount of displacement of the touching hand image to fit to the virtual object, in real time. This modification of the movement of a hand image evokes the effect of the visuo-haptic interaction, and enable users to feel touching various shapes from various viewpoints, although actually users touch a physically static object.

In this study, we developed a system for mobile devices designed to provide a virtual experience of past scenery depicted in old photographs by superimposing them on landscapes in video see-through frames. A user is asked to capture a photograph of a landscape to be used as a keyframe and enter correspondence points between the new and old photos. The old photograph is deformed by projective transform with homography computed based on correspondence points. We then superimpose of the old photograph onto video see-through frames of current landscape. To achieve real-time and robust superimposition on mobile devices, both motion-sensor-based pose information and camera-image-keypoint-tracking-based pose information is used for device's camera pose tracking.

The main objective of this study is to create a method that can affect flavor perception, without changing the food itself, by applying thermal sensations to the skin around the nose to simulate skin temperature changes associated with pleasant and unpleasant feelings. Recent physiological research has demonstrated that skin temperature changes affect flavor perception. Based on the James--Lange theory, which claims that changes in bodily responses unconsciously evoke emotions and change our perception related to an emotional state, we hypothesized that flavor perception can be modified by affecting the skin temperature properly. To test this theory, we developed an "Affecting Tumbler," which presents thermal sensations to the skin around the nose for simulating the skin's temperature response during drinking. Our user study suggested that flavor richness and aftertaste strength were significantly improved by heating up the skin in the nasal region. These results indicate that flavor perception can be controlled with the proposed method.

This study develops a method for improving creativity by changing the facial appearance of people during video conferences. We focus on enhancing creativity during human interactions, especially cooperative work situations such as video conferences. Psychological studies have revealed that the facial appearance of people can affect creativity, i.e., "emotion can affect creativity" and "the appearance of others can affect the emotions." Based on the knowledge, we hypothesized that the media technology which modifies the facial appearance of people appropriately can enhance our creativity during cooperative work. Therefore, we develop a method to affect creativity by changing the facial appearance in real-time, such as facial expressions and facial resemblance. We test the effectiveness of this method by building a system for collaborative video conferences. The user study show that the proposed method could enhance creativity.

In this paper, we proposed a visuo-haptic system for displaying various shapes which have curve, edge, and inclined surfaces, using a simple transmutative physical device and the effect of visuo-haptic interaction. We aim to construct a perception-based shape display system to provide users with the sensation of touching virtual objects of varying shapes using only a simple mechanism. We have confirmed that the perception of each primitive shape such as curvature and angle could be modified by displacing a user's hand image on the monitor as if s/he were touching the visual shape while actually touching another shape. In this study, we constructed the method to merge these findings for displaying more various shapes, including angular ones. We built a transmutative device, which the user touches. The device does not undergo significant transformation, but its surface can be slightly bumped in and out, and displayed various shapes with various angles, length and curvature. The results of experimental trials confirmed that our method for displaying each primitive shape can also worked as designed when we combine these findings to display more complex objects using this device which transforms slightly.

The main contribution of this paper is to develop a method for alleviating fatigue during handling medium-weight objects and augmenting our endurance by affecting our weight perception with augmented reality technology. To assist people to lift medium-weight objects without a complex structure or various costs, we focus on the phenomenon that our weight perception during handling objects is affected by visual properties. Our hypothesis is that this illusionary effect in weight perception can be applied to reduce fatigue while handling medium-weight objects without mechatronics-based physical assistance. In this paper, we propose an augmented reality system that changes the brightness value of an object in order to reduce fatigue while handling the object. We conducted two fundamental experiments to investigate the effectiveness of the proposed system. Our results suggested that the system eliminates the need to use excess energy for handling objects and reduces fatigue during the handling task.

This paper reports a reliving video experience system using augmented reality techniques. This system overlays a scene from video material onto the real world and allows users to experience how the camera operator captured the scene by inducing them to move as in the same way as the camera operator did. We propose novel interaction techniques that balance the competing goals of inducing users to move in the same way as the camera operator and allowing them to feel that they are observing freely. The proposed methods implement three induction techniques to induce the user to avoid unintentional movements, and to start moving the device and rotate it appropriately. We implemented this system in an exhibition system that relives the video of old railways and exhibited it at the railway museum for two weeks. The analysis results of users' operational logs and questionnaires suggest that the proposed system and interaction techniques are effective for reliving video experiences in the real world.

The main contribution of this paper is to realize a method for modifying perception of satiety and controlling nutritional intake by changing the apparent size of food with augmented reality. As a possible method for decreasing rates of obesity, we focused on controlling food intake implicitly without any effort.

In our research, we aim to construct the visuo-haptic system which can give users a sense as if they were touching on virtual objects with variety of shapes, using pseudo-haptiec effects. In this paper, we focus on modifying the identification of a position of edges on a object when touching it with a pointing finger, by displacing the visual representation of the user's hand in order to construct a novel visuo-haptic system. We compose a video see-through system, which enables us to change the perception of the shape of an object a user is visually touching, by displacing the visual representation of the user's hand as if s/he was touching the visual shape, when in actuality s/he is touching another shape. We had experiments and showed participants perceived positions of edges that was the same as the one they were visually touching, even though the positions of edges they were actually touching was different. These results prove that the perceived positions of edges could be modified, and even if the ratio of the successive distance between edges is 1 : 1, we can modify the perception of this ratio from 3 : 2 to 2 : 3.

The main contribution of this paper is to realize computer generated augmented flavors and establish a method to integrate gustatory information into computer human interactions. There are several reasons for the scarcity of research on gustatory information. One reason is that taste sensations are affected by a number of factors, such as vision, olfaction and memories. This produces a complex cognition mechanism for a user's gustatory sensation, and makes it difficult to build up a gustatory display which produces a specific taste on demand.

The purpose of this research is to support users' mealtime communication by changing their utterance rate during meals. By logging and analyzing the utterance rate, or the ratio of utterance length to mealtime length, we determined the relationship between the utterance rate and the distribution of the community to which each meal companion belonged. Using this relationship, we developed a real-time feedback system that presented a pre-estimated utterance rate with rate at the present time during a meal. This utterance rate was estimated according to logged utterance rate data and input data about members at the table. In order to evaluate our system, we asked six users to use it. As a result, we found out that users almost always wanted to increase the utterance rate and users' utterance rates increased as they wanted after using our feedback system.

This study tries to increase the effects of odors by controlling the odor emissions. We developed an olfactory feedback system to maintain user's concentration effectively based on biological information. The system presumes the level of user's concentration based on an electrocardiogram analysis. The analysis is carried out using variance of R-R intervals and frequency spectra. According to the presumed concentration level, the system presents an odor with an awakening effect to maintain user's concentration. We performed experiments in which subjects did simple tasks with this system, changing the method of controlling odor emissions. We compared the results of each method and evaluated the effectiveness of our olfactory feedback system and the methods of controlling odor emissions. The comparison of the results showed that presenting an odor when a user loses his/her concentration is effective to decrease errors of addition tasks.

In this paper, we propose a pseudo-gustatory display based on the cross-modal interactions that underlie the perception of flavor. Although several studies on visual, auditory, haptic, and olfactory displays have been conducted, gustatory displays have seldom been studied. This scarcity has been attributed to the fact that synthesizing arbitrary taste from basic tastants is difficult. On the other hand, it has been noted that the perception of taste is influenced by visual cues, auditory cues, smell, the trigeminal system, and touch. In our research, we aim at utilizing this influence between modalities for realizing a "pseudo-gustatory" system that enables the user to experience various tastes without changing the chemical composition of foods. Based on this concept, we built a "Meta Cookie" system to change the perceived taste of a cookie by overlaying visual and olfactory information onto a real cookie which an AR marker pattern. We performed an experiment that investigates how people experience the flavor of a plain cookie by using our system. The result suggests that our system can change the perceived taste based on the effect of the cross-modal interaction of vision, olfaction and gustation.

In our research demonstration, we show a "MetaCookie+" that enables the user to experience various tastes without changing the chemical composition of foods by using the influence between modalities. It is a pseudo-gustatory display by combining the Edible Marker system which can detect the state [number/shape/6-DOF coordinate] of each piece of bitten or divided food in real time, and the "Pseudo-gustation" method to change the perceived taste of food by changing its appearance and scent.

In this research study, we propose a divided planar-object detection method for augmented reality(AR) applications. There are mainly two types of camera-registration methods for AR applications: marker-based methods, and natural-feature-based methods. In addition, the latter methods are classified into visual SLAM and object detection methods. With respect to object detection methods, particularly for planar objects such as paper, methods for dealing with bending, folding, and occlusion are proposed. However, the division of objects has not been studied. Once an object is divided, a conventional object detection method cannot identify each of the pieces because the feature points of only a single piece are recognized as the target object, and the other feature points are regarded as outliers. The proposed system prepares a database of the target object's natural features, and applies progressive sample consensus(PROSAC), which is a robust estimation method, for iterative homography calculation to achieve the multiple planar-object detection. Moreover, the proposed method can detect shapes of pieces by simultaneously using an occlusion detection method. We demonstrate that it is possible to interact with an arbitrarily divided planar object in real time by our method to implement some AR applications.

In this research study, we propose a divided planar-object detection method for augmented reality(AR) applications. The proposed system prepares a database of the target object's natural features, and applies progressive sample consensus (PROSAC), which is a robust estimation method, for iterative homography calculation to achieve the multiple planar-object detection. Moreover, the proposed method can detect shapes of pieces by simultaneously using an occlusion detection method. We demonstrate that it is possible to interact with an arbitrarily divided planar object in real time by our method to implement some AR applications.

In this paper, we proposed the Digital Diorama system to convey background information vividly. The system superimposes computer generated diorama scene reconstructed from related image/video materials on real exhibits. In order to switch and superimpose real exhibits and past photos seamlessly, we implement a matching system for estimating the camera position where photos are taken. By applying this subsystem to 26 past photos about the steam locomotive exhibit, we succeeded in estimating their camera position. Thus, we implement and install a prototype system at estimated position to superimposing virtual scene and real exhibit in the Railway Museum.

We have developed a novel automultiscopic display, CAGRA (Computer-Aided Graphical Real-world Avatar), which can provide full parallax both horizontally and vertically without imposing any other additional equipment such as goggles on users. CAGRA adopts two axes of rotation so that it can distribute reflected light all around. In our previous work, it was proved that the display can present a black and white image of a 3D object at 1 Hz with both horizontal and vertical parallax. Here, we discuss two important problems that remain unsolved in our previous work: optimal parameter setting and the rendering method. As for finding optimal parameters, the relationship among the rotation speed of two axes and the diffusion angle of the holographic diffuser is discussed. The rendering method was implemented so that accurate view is presented in spite of the unique mechanical structure of CAGRA. Furthermore, the synchronization mechanism between projection of images and rotation of the mirror, which was also a remaining problem, is also implemented. Findings obtained through solving these problems clarify the characteristics of this novel display system, CAGRA, and lead to its further development.

Olfactory displays which exist now can only present the set of scents which was prepared beforehand because a set of "primary odors" has not been found. In this paper, we focus on development of an olfactory display using cross modality which can represent more patterns of scents than the patterns of scents prepared. Due to cross modal effect between vision and olfaction, human tends to feel not olfactory but visual stimulation as scents. First, we asked subjects to smell various aroma chemicals and evaluate their similarity. Based on the data of similarity among aromas, we built a map of smell distance. Next, we selected a few aroma chemicals from the smell distance map and implemented a visual and olfactory display. Then we conducted an assessment experiment of the display. We presented various pictures and the selected aroma chemicals to subjects and asked them what kind of scent they smelled like. In this experiment, we succeed in giving subjects feeling more patterns of scents than the number of selected aroma chemicals. Also, we succeed in making not olfactory stimulation by aroma chemicals but visual stimulation by pictures as scents. In particular, we find that the visual effect on olfactory sensation is more strong when the distance between the picture and the aroma chemicals is close than when the distance is far.

Controllable particle display has been proposed, that controls the position and blinking patterns to yield a visual representation and the representation to touch these particles. Additionally, as an example of its implementation, a controllable particle display using water drops as the particles was proposed. In this system, objects are represented by projecting images upward onto falling water drops designed to form a plane surface, depending on the positions of the water drops. However, this method has a problem in terms of the brightness of the object. In this paper, we propose a method by which images are projected onto falling water drops at an angle, and users observe the images from in front of a projector.

Several indoor positioning systems have been studied to offer a public service based on one's location information. We have studied an indoor localization system using original fiducial markers. The original markers can be designed freely by users; therefore, they can be used for interior decoration. Using this system, you can obtain the three-dimensional position and pose of your camera by capturing the image of the markers arranged on the floor. However, there are some problems associated with the use of this system in a public space, such as a decline in the marker recognition rate by a change in the surrounding light condition and an instability in the marker recognition rate depending on the type of marker used and the camera angle. In this study, we enhanced the robustness of an indoor localization system used in a public space and increased the number of recognizable markers.

We present a novel approach for constructing a virtual 3D world from a sparse set of 2D photograph images. In our approach, we do not construct a large 3D world directly from the images, instead we construct several 3D models from the images and inter-connect them. Each 3D model is photograph-based 3D model that constitutes a few faces and a high-resolution photographic texture image. By arranging some photograph-based 3D models in a 3D scene, we construct a virtual 3D world. To inter-connect these 3D models and represent a unified 3D world, we render the 3D models by blending them together according to view-point position and rotation. Using our novel approach, it is possible to semi-automatically construct a virtual 3D world from fewer photograph images.

Because marker-based position tracking system is inexpensive and easy to use, it has the potential to make the system more feasible for homes or businesses and broaden the current suite of augmented reality (AR) techniques. To apply marker-based systems to homes or businesses, blending markers with environments naturally is important. Our innovative approach to marker-based 3D position tracking uses seamless patterns encrypted with positional data. Although users can obtain 3D positional data by processing marker images similar to many existing marker-based systems, our markers are designed with interior decoration in mind. That way they can be installed in walls, floors, or ceilings. Unlike existing systems whose fiducial markers were designed first and foremost to be processed by computers, ours are visually attractive. By integrating positional information within the interior design, our system enables users to enjoy the benefits of position tracking without being constantly aware of the system's presence. We developed a method for making patterns in which positional information is encrypted and a method for calculating the 3D position of a user by decoding those patterns. We then constructed a system using three patterns that were made by the proposed method and evaluated that system

Recent development in mobile and ubiquitous computing technology with broadband wireless network has realizing the location based application for mobile and wearable computers. According to the wide spreading number of cellular phone with GPS receiver, it has become easy to provide location based application in outdoor scene. On the contrary, there are several positioning technologies for indoor scene, such using wireless sensors or image processing technology. However, those technologies are still not utilized in daily environment due to the cost inefficiency and the complexity in system setup. Considering the problems in previous positioning system, it is necessary to overcome the following four features to utilize the system in daily scene.

Recently, the field of media art has become popular. However, most existing media art works are not suitable to exhibit in a public space. To develop technologies that enable media art works to be exhibited in a public space, we pay attention to the "spatial coexistence between the real and the virtual in public spaces"; the audience in a public space can feel as if the virtual space and real space exist together completely. In this paper, we propose a new concept, named the "controllable particle display", which is to display threedimensional objects by filling space with small particles. On the basis of this concept we developed a prototype system using water drops as particles. In this system, a set of water drops, falling from a tank, are designed to form a plane surface. Patterns of images are projected upward on the falling water drops by a projector under the water drops. Three-dimensional objects can be observed by projecting a set of tomographic images in accordance with the position of the water drops. We also demonstrated the effectiveness of our concept and system.

Recently, there are various types of display systems that can present aural, visual and haptic information related to the user’s position. It is also important to present olfactory information related to the user’s position, and we focus on the spatiality of odor, which is one of its characteristics. In this research, we constructed and evaluated a wearable olfactory display to present the spatiality of odor in an outdoor environment. The prototype wearable olfactory display system treats odor in the gaseous state, and the odor air is conveyed to the user’s nose through tubes. Using this system, we also present the spatiality of odor by controlling the odor strength according to the positions of the user and the odor source. With this prototype system, the user can specify the position of the odor source in an outdoor environment. To improve this prototype system, we constructed another wearable olfactory display. Because odor is treated in the gaseous state, the first prototype system has some problems such as the large size of the device and unintentional leakage of the odor into the environment. To solve these issues, we developed and evaluated an advanced wearable olfactory display that uses an inkjet head device to treat odor in the liquid state.

In this paper, we propose a new approach to building a virtual environment by using raw data from a real environment. To manage and analyze different types of datasets from many sensors and databases, we build and integrate different types of environments with different types of datasets, user interactions and rendering methods. To integrate the environments, we merge rendered images of each virtual environment according to the user's viewpoint, depth information, and reliability of rendered images, and display the resultant image to the user. In addition, we propose an integration environment by using an environment manager, which determines the user's viewpoint with respect to the integrated environments according to the user's input, and compose a display image from rendered images of the integrant environments. The reliability of the rendered images is determined by data accuracy and resolution, display resolution and rendering errors from rendering methodologies. By integrating four constructed environments according to the user's viewpoint, we can construct a photorealistic large-scale environment from different types of datasets. Furthermore, we implemented the prototype system in CABIN and demonstrated the scalability of this architecture.

A worldwide photorealistic environment includes huge amounts of datasets of different types and scales. It is almost impossible to describe such an environment using a single methodology. To manage different types and scales of these datasets, we build and handle virtual subenvironments of different types and scales separately. In addition, we propose an image-based combination method to switch between these local environments smoothly according to the user's viewpoint. We implemented this prototype system in ImmersivProjection Technology (IPT) and demonstrated the efficiency of the method.