This paper aims to verify whether a person’s anger is suppressed when he or she sees a robot that expresses excessive anger in the direction of the same anger factor as the person feels. Social problems caused by human anger, such as aggressive driving and harassment of customers, were increasing. Therefore, the importance of “anger control,” psychological training to understand, manage, and control anger, was growing. This study focuses on the phenomenon where observing others exhibit excessive anger toward the same target reduces one’s own anger when feeling angry. In situations where people feel angry, this study proposes a new anger control method utilizing robots that express stronger anger than humans. Through experiments, the study analyzes the impact of robot anger expressions on users’ emotional states and verifies their effectiveness and potential for anger control.

As a pivotal application of Augmented Reality (AR) technology, AR games empower players to bridge reality with virtuality, offering a distinct and immersive experience set apart from traditional games. However, when creating AR games, one of the most formidable challenges faced by designers pertains to the unpredictability of intricate real-world environments, which hinders crafting naturally integrated scenes where virtual objects harmoniously blend with the players' surroundings. In this paper, we introduce EverywhereAR, a system that is capable of flexibly realizing the designer's idea in various real-world scenes. It provides a designer-friendly Game Scene Template development interface, for designers to quickly graphify their inspirations. To achieve the best AR game scene, this work proposes a highly customizable integration method. According to the integrated AR scene graph, the system will arrange each virtual object in a reasonable position to make the generated game scene look natural. We conducted an experiment to evaluate our system's performance across various game scene templates and real-world environments. Results from the experiment indicated that our system was able to generate AR game scenes matching the quality of scenes manually created by professional designers. In addition, we conducted another experiment to assess the effectiveness and usability of the proposed interface. The experiment results showed that the interface was intuitive and efficient, allowing users to create a simple game scene within one minute.

Performing tasks in a close range using augmented content or instructions visualized on a 2D display can be difficult because of missing visual information in the third dimension. This is because the world on the screen is rendered from the perspective of a single camera, typically on the device itself. However, when performing tasks using hands, haptic feedback supports vision, and prior knowledge and visual context affect task performance. This study rendered the world on a display from the user's perspective to re-enable depth cues from motion parallax and compared it with the conventional device perspective during haptic interactions. We conducted a user study involving 20 subjects and two experiments. First, the accuracy of touchpoint and depth estimation was measured under the conditions of a visual context and perspective rendering on a magic-lens display. We found that user-perspective rendering slightly improved the touch accuracy of targets on a physical surface; however, it significantly improved interactions without tactile feedback. This effect is relatively large when contextual information from the environment is absent, and it diminishes with increased haptic interactions. In the second experiment, we used a user-perspective magic lens to validate the proposed method in a practical needle injection scenario and confirm that the initial injections to virtual targets were more accurate. The results indicate that user-perspective rendering on magic lenses improves immediate performance in haptic tasks, suggesting they are particularly advantageous for frequently changing environments or short-duration tasks.

Augmented reality (AR) magic-lens (ML) displays, such as handheld devices, offer a convenient and accessible way to enrich our environment using virtual imagery. Several display technologies, including conventional monocular, less common stereoscopic, and varifocal displays, are currently being used. Vergence and accommodation effects on depth perception, as well as vergence-accommodation conflict, have been studied, where users interact only with the content on the display. However, little research exists on how vergence and accommodation influence user performance and cognitive-task load when users interact with the content on a display and its surroundings in a short timeframe. Examples of this are validating augmented instructions before making an incision and performing general hand-eye coordinated tasks such as grasping augmented objects. To improve interactions with future AR displays in such scenarios, we must improve our understanding of this influence. To this end, we conducted two fundamental visual-acuity user studies with 28 and 27 participants, while investigating eye vergence and accommodation distances on four ML displays. Our findings show that minimizing the accommodation difference between the display and its surroundings is crucial when the gaze between the display and its surroundings shifts rapidly. Minimizing the difference in vergence is more important when viewing the display and its surroundings as a single context without shifting the gaze. Interestingly, the vergence-accommodation conflict did not significantly affect the cognitive-task load nor play a pivotal role in the accuracy of interactions with AR ML content and its physical surroundings.

Augmented Reality (AR) has long been expected to help users improve their working efficiency. However, due to the absence of intelligent systems, existing AR applications are greatly affected by the virtual content interference with real-world activities. Unlike existing work, which focuses more on hiding virtual content to reduce interference, in this work, we propose an innovative AR Task Support System where virtual contents actively guide users with task completion. During task execution, our system proactively searches for and tracks key objects in the scene, and uses this context information to automatically select appropriate virtual content and display positions. Through introducing open-world prompt-based visual models, our system can effectively retrieve few-shot or even zero-shot objects that are uncommon in the dataset. This approach extends the use of AR Task Support System beyond controlled industrial settings to more uncontrolled daily scenarios, overcoming the limitations of existing systems. It also significantly reduces development costs for developers. We demonstrate the advantages of our system over traditional virtual content management systems through a series of experiments that are closer to users' real usage situations.

The use of robots that interact with humans in social services has been attracting attention, and the rapid development of AI for automatic dialogue generation has raised expectations for its realization. However, the issue of privacy protection of dialogues between robot and human, which is important when using actual services, has not been taken into account. In this study, we propose a privacy-secure dialogue framework for human-robot interaction (HRI) that comprehensively protects the privacy of dialogue content when a human and a robot engage in dialogue, and we consider privacy protection in all stages from continuous speaker authentication to dialogue generation and dialogue approach. This framework will greatly contribute to the social diffusion of interactive robot services, which are expected to play an active role in public spaces in the future.

Abstract Lenslet array near-eye displays are a revolutionary technology that generates a virtual image in the field of view of the observer. Although this technology is advantageous in creating compact near-eye displays, undesirable artifacts occur when the user pupil moves outside of the pupil practical movable region (PPMR). Even with dynamic image updating based on eye-tracking techniques, artifacts can still be perceived when human eyes turn rapidly. To enlarge PPMR, we proposed a new rendering method in previous work. To improve the rendering speed in the eye tracking system, look-up tables are used. The disadvantage of the onboard system is the large memory consumption. In this study, we analyzed the system parameters of the incident pupil and pupil margin light columns, the feasibility of the optimized system, and evaluated the optimized system that can adapt to the maximum velocity of the saccadic pupil movement. We optimized the rendering method to reduce memory consumption in the process of generating microdisplay images. In addition, we provide GPU rendering method to improve system speed and reduce system latency to meet the maximum human eye rotation speed. We conducted user studies to evaluate the effect of the method using the optimized rendering method combined with eye tracking to reduce artifacts for fast eye rotation on different images and videos. Results showed that our method effectively reduced artifacts via the optimized rendering method with eye tracking, which adapted to faster human eye movements.

This study proposes a communication robot that plays video games together with the user to create a fun atmosphere for a daily-use robot. In recent years, the lack of conversation in daily life has become a problem, and as a solution to this problem, a dialogue robot have attracted attention as a substitute for a human and as a means of encouraging daily speech. In this study, we propose a dialogue robot that plays a video game with a user in order to produce enjoyable daily communication with the robot. The proposed method not only controls the robot’s speech, but also controls the game development by manipulating the game characters. The proposed robot enhances a user’s enjoyment by adjusting the level of game manipulation to produce a competitive match and by speaking according to the game scene. A robot that produces speech that makes a player feel positive emotions is called a friendly robot. In order to examine the effects of these robots on a user’s enjoyment, we conducted an experiment with 30 subjects in which they were subjectively evaluated by questionnaires in two types of matches: with a friendly robot and alone. The results showed that robots were more enjoyable than playing alone.

Previous research has shown that many factors in a virtual environment influence the way, how we perceive interaction with humanoid agents. It is still unknown if visual stereotypes also translate into human-robot interaction scenarios. To investigate this relationship, we conducted a two-part user study with 26 participants using the Stereotype Content Model (SCM) to identify different stereotypical avatars and visual features in terms of their effect on the perception of robot touch. We found that humanoid avatars which resemble the robot and are perceived as warm in the SCM are more liked and preferred than cold avatars. We also found different effects of visual appearance on touch perception. This suggests that a humanoid-stereotpyical appearance of a touch-care robot in virtual reality can change the overall perception of the interaction. We propose the use of stereotypical warm avatars as described in the SCM for humanity-based applications.

In this study, we analyzed the human "gentle stroke with speech behavior" for comfortable motion by implementing the model into a robot. The results of 19 subjects’ human analysis showed that there was a positive correlation between stroke speed and speech speed. The faster the speech speed, the larger the variance of each feature. Moreover, it was also estimated regardless of the length of each sentence, people tend to fold their hands at "breaks in sentences" or "points just before breaks in sentences". Based on this human-modeled knowledge, we evaluated a human comfort experiment in 36 subjects using a robot arm to reproduce the "stroke and speech" behavior at different speeds. Results of the questionnaire showed that subjects felt more comfortable when stroke speed and speech speed were positively correlated. This result suggests that people consciously adjust their stroking and speaking styles when they perform "comfortable stroke with speech behavior".

Recent augmented reality (AR) advancements have enabled the development of effective training systems, especially in the medical, rehabilitation, and industrial fields. However, it is unclear from the literature what the intrinsic value of AR to training is and how it differs across multiple application fields. In this work, we gathered and reviewed the prototypes and applications geared towards training the intended user's knowledge, skills, and abilities. Specifically, from IEEE Xplore plus other digital libraries, we collected 64 research papers present in high-impact publications about augmented reality training systems (ARTS). All 64 papers were then categorized according to the training method used, and each paper's evaluations were identified by validity. The summary of the results shows trends in the training methods and evaluations that incorporate ARTS in each field. The narrative synthesis illustrates the different implementations of AR for each of the training methods. In addition, examples of the different evaluation types of the current ARTS are described for each of the aforementioned training methods. We also investigated the different training strategies used by the prevailing ARTS. The insights gleaned from this review can suggest standards for designing ARTS regarding training strategy, and recommendations are provided for the implementation and evaluation of future ARTS.

Nowadays, decreasing opportunities for people to have daily conversations due to the increase in the number of withdrawn young people and households living alone. The lack of daily conversation has been pointed out as a risk that can lead to mental problems such as depression, and serious health problems such as dementia for the elderly. Efforts to encourage daily communication by having communication robots that act as talking partners is attracting attention to solve those problems. One of the challenges of communication robots is the difficulty of maintaining users' motivation to continue using robots. In this study, we propose the communication robot that plays a video game together with a user as an approach to keep the user's motivation high to use the robot. The proposed method not only controls the dialogue content of the robot but also controls the video game situation by manipulating the video game character. In this system, we aim to create an atmosphere where users can enjoy playing video games with the robot together, and the proposed game communication robot can keep their motivation high to use the robot.

Projector-camera systems have long been used in measuring three-dimensional shapes. Most projector-camera systems can only be used in dark rooms because frame-based cameras are not robust against strong ambient light and are difficult to obtain correspondence to the image pixels of the projector. Recently, event cameras, which can detect the direction of luminance change, have received attention in the field of computer vision. When considering the many advantages of event cameras, this study focuses on their wide dynamic range (120 vs. 40 dB of a frame-based camera) and their ability to detect fast luminance changes. Our objective is to realize a projector-camera system that combines the event camera with a projector under the strong ambient light. Specifically, this study proposes a new structured light that combines different frequencies of flickers to acquire the correspondence between the image pixels of the event camera and the projector. This method does not rely on the co-axial frame-based measurement and synchronization mechanism between projector and camera and is thus applicable to most general event cameras. Experiments confirm that the proposed method obtains the correspondence robustly with reasonable accuracy in a bright room (up to 2,600 lux) under general indoor lighting and additional light projection.

One of the main targets of criticism of head-mounted displays (HMDs) is the field of view (FOV) size, whether in virtual or augmented reality. This limitation is prominent with optical see-through head-mounted displays (OST-HMD), as those with narrow overlay FOV (OFOV) sizes only provide a small window to view virtual objects. We investigated if restricting this OFOV negatively affects a user’s ability to memorize spatial locations in a simulation of a work environment, and consequently, long-term memory transfer to an equivalent scenario in the real world two days later. To find empirical evidence, we conducted a within-subjects experiment with 18 participants performing in three phases with an OST-HMD, simulated on an immersive HMD. For each phase, they viewed the training scenario with a different OFOV size of the augmentable area (30°, 70°, 110° diagonal). Results from recall tests showed that smaller OFOV size did not significantly affect user’s performance on both short-term and transfer tests, but HMD data revealed that users rotated their heads less with a 110° OFOV. We also found that proximity of objects to memorize had an interaction effect with smaller OFOV sizes. Our findings could have implications on the design and HMD choices of augmented training.

Triangle meshes are used in many important shape-related applications including geometric modeling, animation production, system simulation, and visualization. However, these meshes are typically generated in raw form with several defects and poor-quality elements, obstructing them from practical application. Over the past decades, different surface remeshing techniques have been presented to improve these poor-quality meshes prior to the downstream utilization. A typical surface remeshing algorithm converts an input mesh into a higher quality mesh with consideration of given quality requirements as well as an acceptable approximation to the input mesh. In recent years, surface remeshing has gained significant attention from researchers and engineers, and several remeshing algorithms have been proposed. However, there has been no survey article on remeshing methods in general with a defined search strategy and article selection mechanism covering the recent approaches in surface remeshing domain with a good connection to classical approaches. In this article, we present a survey on surface remeshing techniques, classifying all collected articles in different categories and analyzing specific methods with their advantages, disadvantages, and possible future improvements. Following the systematic literature review methodology, we define step-by-step guidelines throughout the review process, including search strategy, literature inclusion/exclusion criteria, article quality assessment, and data extraction. With the aim of literature collection and classification based on data extraction, we summarized collected articles, considering the key remeshing objectives, the way the mesh quality is defined and improved, and the way their techniques are compared with other previous methods. Remeshing objectives are described by angle range control, feature preservation, error control, valence optimization, and remeshing compatibility. The metrics used in the literature for the evaluation of surface remeshing algorithms are discussed. Meshing techniques are compared with other related methods via a comprehensive table with indices of the method name, the remeshing challenge met and solved, the category the method belongs to, and the year of publication. We expect this survey to be a practical reference for surface remeshing in terms of literature classification, method analysis, and future prospects.

Accurately placing virtual objects in a scene is a challenging tasks in handheld augmented reality (HAR). To add and arrange virtual objects in HAR, users must manipulate 6 degrees of freedom (DoFs) of the virtual object, namely: position (3) and orientation (3). However, it is difficult to manipulate all DoFs with the two-dimensional display of the handheld device. We present SlidAR+, a method for controlling the position and orientation of objects in HAR. SlidAR+ is an extension of SlidAR [1], a technique that allows users to control the position of a virtual object by manipulating only 1 DoF. We use the direction of gravity as a constraint to improve the user’s control and reduce the time it takes to adjust the orientation. Upon comparing it with a state-of-the-art object manipulation method, using SlidAR+, user were able to complete the tasks faster under our expected conditions and were also preferred by most participants.

Augmented reality (AR) systems support complex tasks like assembly by overlaying task-related content onto the real world. In recent years, the effort of designing and developing assembly task support systems in AR decreased with the availability of high potential head-mounted displays and provision of integrated development environments. Nevertheless, problems still arise when companies craft an effective AR task support system, particularly in the difficulty of selecting appropriate techniques and information-presentation methods, and the requirements that vary with each use case. In this study, we formulated a corresponding guideline, developed a selection aid tool that incorporates filtering based on the categorization of subtasks and the degree of freedom of available tracking, and evaluated their effectiveness in two experiments. First, to confirm effects on system design, we asked 18 participants to perform the design action of the AR system with the guideline for two tasks (PC assembly and rope work). Consequently, to verify the quality of the designed AR systems from Experiment 1, we asked another set of 20 participants to perform the same tasks with those systems. The results confirm that using the guideline can considerably lower efforts creating media and alleviate the error for a specific process. We envision our guideline and tool to be accessible as an online web page, assisting AR assembly task support system designer/developers worldwide.

With projection mapping, we can control the appearance of real-world objects by adding illumination. A projector can be used to control light reflected from an object, where the reflected light depends not only on the projection but also on the reflectance and environmental light. Because the resulting colors are affected by the reflectance and environmental light, the presentable color range of a projector is limited. The purpose of this work is to broaden this limited range by focusing on the perceived colors. Although our eyes capture reflected light to perceive the colors of an object, the colors perceived by humans are not always the same as the actual colors, and there are often significant differences between them because of the human visual system. To overcome the limitations of a projector based on human perception, we intentionally generate this difference by inducing a visual illusion, namely, color constancy. In this work, we designed an algorithm to determine the projected colors for presenting the desired colors perceptually by employing a color constancy effect. In addition, we conducted a user study and confirmed that our algorithm can (1) create a misperception regarding the color of illumination, (2) broaden the presentable color range of a projector, and (3) shift the perceptual colors in the desirable direction.

Humanitude is a multimodal communication care method that utilizes seeing, touching, and speaking. Moreover, a touch-care method is well known as an effective care method mainly focus on touch motion. These kinds of care techniques are effective in practical situations, however, it is difficult to provide such care therapy to all patients due to the lack of human resources. To address this problem, researchers try to develop a touch-care robot that can provide touch-care automatically. Conventional research of touch-care robot mainly focuses on the movement of stroke or the speech that only considers the impression of the contents of speech but not prosodic information. Therefore, in this research, we focus on the speech rate in the prosodic information with stroke motion. In this work, we investigate the effects of speech rate on the prosodic information and evaluate the relationship between stroke pace and speech rate to improve human comfort. We conducted a user study with 6 participants around 20 years old males. As a result of the list of the questionnaire suggests a correlation between stroke pace and speech rate that provides comfort.

This paper proposes a pacing system that makes dialog robots mimic the pacing technique in order to promote trust relationships between humans and robots. Pacing is a communicative technique that synchronizes breathing or how to speak with a person to build relationships among humans. The experiment was performed by comparing the impression of human subjects' conversation with a robot when the pacing system was applied or not applied. The evaluation used both subjective and objective perspectives. From the subjective perspective, the results show significant differences for the questionnaire items concerning "feeling friendliness," "feeling fun," "feeling an emotional connection," "feeling synchrony," "motivation to use" and "listening to the user" between the with and without pacing conditions. Also, from the objective perspective, there is a significant difference in sympathetic activity in heart rate variability.

This paper presents the effects of being hugged by a robot to encourage prosocial behaviors. In human-human interaction, touches including hugs are essential for communication with others. Touches also show interesting effects, including the “Midas touch,” which encourages prosocial behaviors from the people who have been touched. Previous research demonstrated that people who touched a robot experienced positive impressions of it without clarifying whether being hugged by a robot causes the Midas touch effect, i.e., positively influences engagement in prosocial behaviors. We developed a huge, teddy-bear-like robot that can give reciprocal hugs to people and experimentally investigated its effects on their behaviors. In the experiment, a robot first asked participants to give a hug and then asked them to make charitable donations in two conditions: with or without a reciprocated hug. Our experiment results with 38 participants showed that those who were hugged by a robot donated more money than those who only hugged the robot, i.e., without a reciprocated hug.

In this paper, we experimentally examine the relationship between visual cognition difficulty and target-tracking eye movements, which recorded during moving target cognition. Generally, such eye movements are observed when humans perceive a moving object and they vary widely due to many factors, such as target shape, backgrounds, illumination conditions, and so on. Several systems have been proposed for estimating human cognition based on gaze movements. However, since most of them employ simple thresholding techniques to classify the states of cognition, their classification performance remain insufficient. This research clarifies the relationship between multiple visual conditions and target-tracking eye movements to enhance the classification performance. We found that we can address a variety of target and background factors from the perspective of visual cognition difficulty for the targets. Observed gaze movement properties and the difficulty of target visual cognition have a linear relationship, suggesting the possibility of more precise estimation of human's target cognition based on them.

Personal mobility devices such as wheelchairs, bicycles, and compact cars are used in daily life and to runs on sidewalks. However, there are several factors that may lead to discomfort rides, such as steps, slopes, and crowded sidewalks for a passenger. This paper proposes a system that detects the factors using smartphone attached to the personal mobility device and generates a discomfort-ride map including the environment and human factors and their rating on sidewalks. In the experiment, steps (static danger factor) and the dynamic moving obstacles like human, bicycle, and car (dynamic danger factors) were estimated with attached smartphone's acceleration sensor and camera. After the process of collecting these data, verification for generating the hazard map system based on these data. Moreover, the versatility of using the proposed hazard map system for different personal mobility devices was tested with the electric wheelchair and the bicycle.

This paper presents an approach for motion sickness reduction while riding an autonomous vehicle. It proposes the Diminished Reality (DR) method for an acceleration stimulus to reduce motion sickness for the autonomous vehicle. One of the main causes of motion sickness is a repeated acceleration. In order to diminish the acceleration stimulus in the autonomous vehicle, vection illusion is used to induce the user to make a preliminary movement against the real acceleration. The Balance Wii Board is used to measure participant's movement of the center of gravity to verify the effectiveness of the method with vection. The experimental result of 9 participants shows that the proposed method of using vection could reduce acceleration stimulus compared with the conventional method.

This paper proposes a novel framework to estimate the amount of physical activity at a place where people stayed, by utilizing facility information and user's acceleration data. The total amount of physical activities, energy expenditure of a physical activity is a good scale. Our framework provides a physical activity scale based on typical energy expenditure of the activity given by existing researches already. To estimates the energy expenditure, we use typical value of metabolic equivalents to task (MET) which is used as practical scale. Unlike the other studies for monitoring physical activity, we estimate the type of user's activity using facility information which are obtained from road map and land-use/land-cover map. To confirm the feasibility of our approach, we have conducted a long-term experiment on monitoring the activity of elderly people living in less-populated area. As a result, our framework provides good summarization of daily activities of participants.

This study proposes a framework to photorealistically synthesize virtual objects and virtualized real-world. We combine the offline rendering of virtual objects and the free-viewpoint image generation to take advantage of the higher quality of offline rendering without the computational cost of online computer graphics (CG) rendering; i.e., it incurs only the cost of the online computation for the free-viewpoint image generation. In addition, the generation of structured viewpoints (e.g., at every grid point) reduces the computational costs required to online process.

This paper describes a method for acquiring full spherical high dynamic range (HDR) images with no missing areas by using two omni directional cameras mounted on the top and bottom of an unmanned airship. The full spherical HDR images are generated by combining multiple omni directional images that are captured with different shutter speeds. The images generated are intended for uses in telepresence, augmented telepresence, and image-based lighting.

We have developed an augmented telepresence system which enables virtual tourism beyond time and space. Augmented telepresence provides a user with both the view of a remote location and related information using augmented reality techniques. This study deals with the geometric and photometric registration problems to generate movie-quality augmented omnidirectional videos automatically. The user can look around the scene from the sky above Heijo palace Site which is an ancient capital in Nara, Japan in the technical demonstration.

This paper proposes a novel method for generating arbitrary stereoscopic view from multiple omni directional image sequences. Although conventional methods for arbitrary view generation with an image-based rendering approach can create binocular views, positions and directions of viewpoints for stereoscopic vision are limited to a small range. In this research, we attempt to generate arbitrary stereoscopic views from omni directional image sequences that are captured in various multiple paths. To generate a high-quality stereoscopic view from a number of images captured at various viewpoints, appropriate ray information needs to be selected. In this paper, appropriate ray information is selected from a number of omni directional images using a penalty function expressed as ray similarity. In experiments, we show the validity of this penalty function by generating stereoscopic view from multiple real image sequences.

This study is concerned with a large-scale telepresence system based on remote control of mobile robot or aerial vehicle. The proposed system provides a user with not only view of remote site but also related information by AR technique. Such systems are referred to as augmented telepresence in this paper. Aerial imagery can capture a wider area at once than image capturing from the ground. However, it is difficult for a user to change position and direction of viewpoint freely because of the difficulty in remote control and limitation of hardware. To overcome these problems, the proposed system uses an autopilot airship to support changing user's viewpoint and employs an omni-directional camera for changing viewing direction easily. This paper describes hardware configuration for aerial imagery, an approach for overlaying virtual objects, and automatic control of the airship, as well as experimental results using a prototype system.

In annotation overlay applications using augmented reality (AR), view management is widely used for improving readability and intelligibility of the annotations. In order to recognize the visible portions of objects in the user's view, the positions, orientations, and shapes of the objects should be known in the case of conventional view management methods. However, it is difficult for a wearable AR system to obtain the positions, orientations and shapes of objects because the target object is usually moving or non-rigid. In this paper, we propose a view management method to overlay annotations of moving or non-rigid objects for networked wearable AR. The proposed method obtains positions and shapes of target objects via a network in order to estimate the visible portions of the target objects in the user's view. Annotations are located by minimizing penalties related to the overlap of an annotation, occlusion of target objects, length of a line between the annotation and the target object, and distance of the annotation in sequential frames. Through experiments, we have proven that the prototype system can correctly provide each user with annotations on multiple users of wearable AR systems.

This paper describes a real-time pre-visualization method using augmented reality techniques for videographers. It enables them to test camera work without real actors in a real environment. As a substitute for real actors, virtual ones are superimposed on a live video in real-time according to a real camera motion and an illumination condition. The key technique of this method is real-time motion estimation of a camera, which can be applied to unknown complex environments including natural objects. In our method, geometric and photometric registration problems for such unknown environments are solved to realize the above visualization. A prototype system demonstrates availability of the pre-visualization method.

This paper describes photometric registration for augmented reality (AR) using a high-dynamic-range (HDR) image. In photorealistic AR, estimating the lighting environment of virtual objects is difficult because of low dynamic range cameras. In order to overcome this problem, we propose a method that estimates the lighting environment from an HDR image and renders virtual objects using an HDR environment map. Virtual objects are overlaid in real-time by adjusting the dynamic range of the rendered image with tone mapping according to the exposure time of the camera. The HDR image is generated from multiple images captured with various exposure times. We have found through experimentation that the updating rate is improved by effectively limiting the dynamic range, depending on the exposure time. We have verified the effect of limiting the dynamic range on the reality of virtual objects.

We propose a user localization system that uses invisible markers for wearable augmented reality (AR) in large indoor environments. Wearable AR systems have received a great deal of attention as a new method for displaying location-based information in the real world. For using wearable AR systems, it is necessary to measure the position and orientation of a user using a positioning infrastructure without the undesirable visual effects that arise from merging real and virtual worlds. In addition, the infrastructure of the localization environment must be constructed easily and cheaply. The proposed system can estimate the position and orientation of a user precisely by affixing wallpapers containing printed invisible markers on ceilings or walls. The user's position and orientation are estimated by recognizing the markers using an infrared camera with infrared LEDs. To construct environments for the localization system, we developed an initialization tool that calibrates the alignment of the markers from photographs taken by flash illumination using a digital still camera.

This paper proposes an accurate geometric registration method by estimating blur effects from a degraded image with image markers for augmented reality. A small and inexpensive camera used in augmented reality systems sometimes captures degraded images because its focus and/or iris are fixed. This degradation of a captured image affects the accuracy of the detected positions of feature points in the image. The proposed method improves the accuracy of the estimated camera position and posture by estimating blur effects from the captured image, and by correcting the detected positions of feature points through the results. The effectiveness of the method is confirmed through experiments of corner estimation from simulated images and extrinsic camera parameter estimation from real images.

Photometric registration is as important as geometric registration to generate a seamless augmented reality scene. Especially the difference in image quality between a real image and virtual objects caused by defocusing and motion blurring in capturing a real scene image easily exhibits the seam between real and virtual worlds. To avoid this problem in video see-through augmented reality, it is necessary to simulate the optical system of camera when virtual objects are rendered. This paper proposes an image composition method for video see-through augmented reality, which is based on defocusing and motion blurring estimation from the captured real image and rendering of virtual objects with blur effects. In experiments, the effectiveness of the proposed method is confirmed by comparing a real image with virtual objects rendered by the proposed method.

By realizing augmented reality on wearable computers, it is possible to overlay annotations on the real world based on the user's current position and orientation. However, it is difficult for the user to understand links between annotations and real objects intuitively when the scene is complicated or many annotations are overlaid at the same time. This work describes a view management method which emphasizes user-viewed real objects and their annotations using 3D models of the real scene. The proposed method highlights the objects viewed by the user. In addition, when the viewed object is occluded by other real objects, the object is complemented by using an image, which is made from 3D models, on the overlaid image.

This paper describes a method for modeling wide area outdoor environments by integrating omnidirectional range and color images. The proposed method effectively reconstructs the 3D models of outdoor environments by using omnidirectional laser rangefinder and omnidirectional multi-camera system (OMS). In this paper, we also give experimental results of 3D wide area reconstruction using the data acquired at 50 points in our campus.

This paper describes a novel telepresence system that uses high-resolution movies and a reactive display system with a treadmill. In this system, users can walk through a virtualized environment by actually walking on a treadmill. According to walking motion which is detected by using 3-D position sensors put on both legs, the virtualized environment captured by an omnidirectional multi-camera system is projected on a multi-screen display.

This paper describes a wearable augmented reality system using positioning infrastructures and a pedometer. To realize augmented reality systems, the position and orientation of user's viewpoint should be obtained in real time. The proposed system measures the orientation of user's viewpoint by an inertial sensor and the user's position using positioning infrastructures in environments and a pedometer. The system specifies the user's position using the position ID received from RFID tags or IrDA markers which are the components of positioning infrastructures. When the user goes away from them, the user's position is alternatively estimated by using a pedometer. We have developed a navigation system using the proposed techniques and have proven the feasibility of the system with experiments.

In vision-based augmented reality systems, the relationship between the real and virtual worlds needs to be estimated to perform the registration of the virtual objects. This paper suggests a registration method which increases the registration depth of video see-through augmented reality systems using binocular cameras. The method uses both monocular and stereoscopic vision-based techniques in order to perform the registration. Also, the registration method can be combined with a facultative correction of the 2D positions of the feature points. The correction increases the stability and the accuracy of the proposed registration method.

This paper proposes an augmented reality system with correct representation of shading and shadow. To realize a seamless augmented reality system, we need to resolve certain problems. The geometric and photometric registration problems are particularly important. These problems require the position of light sources and user's viewpoint. The proposed system resolves these problems using a 3D marker which combines a 2D square marker and a mirror ball. The 2D marker and the ball are used to estimate the relationship between the real and virtual worlds and the positions of light sources in the real world, respectively.

Proposes a vision-based augmented reality system with a wide range of registration. To realize an augmented reality system, it is required to geometrically register real and virtual worlds. In the case of a vision-based augmented reality with marker tracking, its measurement range is usually limited because markers placed in the real world should be captured by cameras. The proposed method realizes a stereo vision-based augmented reality system with a wide range of registration by automatically detecting and tracking new, markers that come into sight. The feasibility of the system has been successfully demonstrated through experiments.

In an augmented reality system, it is required to obtain the position and orientation of the user's viewpoint in order to display the composed image while maintaining a correct registration between the real and virtual worlds. All the procedures must be done in real time. This paper proposes a method for augmented reality with a stereo vision sensor and a video see-through head-mounted display (HMD). It can synchronize the display timing between the virtual and real worlds so that the alignment error is reduced. The method calculates camera parameters from three markers in image sequences captured by a pair of stereo cameras mounted on the HMD. In addition, it estimates the real-world depth from a pair of stereo images in order to generate a composed image maintaining consistent occlusions between real and virtual objects. The depth estimation region is efficiently limited by calculating the position of the virtual object by using the camera parameters. Finally, we have developed a video see-through augmented reality system which mainly consists of a pair of stereo cameras mounted on the HMD and a standard graphics workstation. The feasibility of the system has been successfully demonstrated with experiments.