This study investigated the effect of biomimetic feeding behavior on the perception of lifelikeness in robots. A robot designed to simulate the feeding behavior of the emperor penguin was developed. It features a 3D-printed beak, movable wings, and a furry exterior, which allows for safe tactile interaction. Participants interacted with the robot in two conditions: with and without feeding. The results indicate that feeding may enhance the lifelikeness and perceived intelligence of the robot. This outcome suggests the potential for interaction through feeding to facilitate a deeper connection between humans and robots, which will be useful for designing more lifelike robots in the future.

The number of studies examining the potential of immersive entertainment in automated vehicles is increasing. It has been demonstrated that synchronizing the visual content of virtual reality (VR) with the movements of the vehicle and matching the passenger's visual and experiential senses can enhance the immersive experience. In consideration of the preceding studies, this study proposes a novel immersive automatic driving system that prioritizes the vehicle's behavior during acceleration. The objective is to enable passengers to perceive a sense of hyper-acceleration that exceeds the actual acceleration, thereby facilitating a more immersive experience in automatic driving VR entertainment through the manipulation of vehicle behavior.

Flood risk communication in disaster-prone communities often relies on traditional tools (e.g., paper and browser-based hazard/flood maps) that struggle to engage community stakeholders and reflect intuitive flood situations. In this paper, we applied the transitional mixed reality (MR) interface concept from pioneering work and extended it for flood risk communication scenarios through co-design with community stakeholders to help vulnerable residents understand flood risk and facilitate preparedness. Starting with an initial transitional MR prototype, we conducted three iterative workshops - each dedicated to device usability, visualization techniques, and interaction methods. We collaborated with diverse community stakeholders in flood-prone areas, collecting feedback to refine the system according to community needs. Our preliminary evaluation indicates that this co-designed system significantly improves user understanding and engagement compared to traditional tools, though some older residents faced usability challenges. We detailed this iterative co-design process, critical insights and design implications, offering our work as a practical case of mixed reality application in strengthening flood risk communication. We also discuss the system's potential to support community-driven collaboration in flood preparedness.

As a pivotal application of Augmented Reality (AR) technology, AR games empower players to bridge reality with virtuality, offering a distinct and immersive experience set apart from traditional games. However, when creating AR games, one of the most formidable challenges faced by designers pertains to the unpredictability of intricate real-world environments, which hinders crafting naturally integrated scenes where virtual objects harmoniously blend with the players' surroundings. In this paper, we introduce EverywhereAR, a system that is capable of flexibly realizing the designer's idea in various real-world scenes. It provides a designer-friendly Game Scene Template development interface, for designers to quickly graphify their inspirations. To achieve the best AR game scene, this work proposes a highly customizable integration method. According to the integrated AR scene graph, the system will arrange each virtual object in a reasonable position to make the generated game scene look natural. We conducted an experiment to evaluate our system's performance across various game scene templates and real-world environments. Results from the experiment indicated that our system was able to generate AR game scenes matching the quality of scenes manually created by professional designers. In addition, we conducted another experiment to assess the effectiveness and usability of the proposed interface. The experiment results showed that the interface was intuitive and efficient, allowing users to create a simple game scene within one minute.

Performing tasks in a close range using augmented content or instructions visualized on a 2D display can be difficult because of missing visual information in the third dimension. This is because the world on the screen is rendered from the perspective of a single camera, typically on the device itself. However, when performing tasks using hands, haptic feedback supports vision, and prior knowledge and visual context affect task performance. This study rendered the world on a display from the user's perspective to re-enable depth cues from motion parallax and compared it with the conventional device perspective during haptic interactions. We conducted a user study involving 20 subjects and two experiments. First, the accuracy of touchpoint and depth estimation was measured under the conditions of a visual context and perspective rendering on a magic-lens display. We found that user-perspective rendering slightly improved the touch accuracy of targets on a physical surface; however, it significantly improved interactions without tactile feedback. This effect is relatively large when contextual information from the environment is absent, and it diminishes with increased haptic interactions. In the second experiment, we used a user-perspective magic lens to validate the proposed method in a practical needle injection scenario and confirm that the initial injections to virtual targets were more accurate. The results indicate that user-perspective rendering on magic lenses improves immediate performance in haptic tasks, suggesting they are particularly advantageous for frequently changing environments or short-duration tasks.

Augmented reality (AR) magic-lens (ML) displays, such as handheld devices, offer a convenient and accessible way to enrich our environment using virtual imagery. Several display technologies, including conventional monocular, less common stereoscopic, and varifocal displays, are currently being used. Vergence and accommodation effects on depth perception, as well as vergence-accommodation conflict, have been studied, where users interact only with the content on the display. However, little research exists on how vergence and accommodation influence user performance and cognitive-task load when users interact with the content on a display and its surroundings in a short timeframe. Examples of this are validating augmented instructions before making an incision and performing general hand-eye coordinated tasks such as grasping augmented objects. To improve interactions with future AR displays in such scenarios, we must improve our understanding of this influence. To this end, we conducted two fundamental visual-acuity user studies with 28 and 27 participants, while investigating eye vergence and accommodation distances on four ML displays. Our findings show that minimizing the accommodation difference between the display and its surroundings is crucial when the gaze between the display and its surroundings shifts rapidly. Minimizing the difference in vergence is more important when viewing the display and its surroundings as a single context without shifting the gaze. Interestingly, the vergence-accommodation conflict did not significantly affect the cognitive-task load nor play a pivotal role in the accuracy of interactions with AR ML content and its physical surroundings.

Augmented Reality (AR) has long been expected to help users improve their working efficiency. However, due to the absence of intelligent systems, existing AR applications are greatly affected by the virtual content interference with real-world activities. Unlike existing work, which focuses more on hiding virtual content to reduce interference, in this work, we propose an innovative AR Task Support System where virtual contents actively guide users with task completion. During task execution, our system proactively searches for and tracks key objects in the scene, and uses this context information to automatically select appropriate virtual content and display positions. Through introducing open-world prompt-based visual models, our system can effectively retrieve few-shot or even zero-shot objects that are uncommon in the dataset. This approach extends the use of AR Task Support System beyond controlled industrial settings to more uncontrolled daily scenarios, overcoming the limitations of existing systems. It also significantly reduces development costs for developers. We demonstrate the advantages of our system over traditional virtual content management systems through a series of experiments that are closer to users' real usage situations.

Abstract Lenslet array near-eye displays are a revolutionary technology that generates a virtual image in the field of view of the observer. Although this technology is advantageous in creating compact near-eye displays, undesirable artifacts occur when the user pupil moves outside of the pupil practical movable region (PPMR). Even with dynamic image updating based on eye-tracking techniques, artifacts can still be perceived when human eyes turn rapidly. To enlarge PPMR, we proposed a new rendering method in previous work. To improve the rendering speed in the eye tracking system, look-up tables are used. The disadvantage of the onboard system is the large memory consumption. In this study, we analyzed the system parameters of the incident pupil and pupil margin light columns, the feasibility of the optimized system, and evaluated the optimized system that can adapt to the maximum velocity of the saccadic pupil movement. We optimized the rendering method to reduce memory consumption in the process of generating microdisplay images. In addition, we provide GPU rendering method to improve system speed and reduce system latency to meet the maximum human eye rotation speed. We conducted user studies to evaluate the effect of the method using the optimized rendering method combined with eye tracking to reduce artifacts for fast eye rotation on different images and videos. Results showed that our method effectively reduced artifacts via the optimized rendering method with eye tracking, which adapted to faster human eye movements.

This study proposes a communication robot that plays video games together with the user to create a fun atmosphere for a daily-use robot. In recent years, the lack of conversation in daily life has become a problem, and as a solution to this problem, a dialogue robot have attracted attention as a substitute for a human and as a means of encouraging daily speech. In this study, we propose a dialogue robot that plays a video game with a user in order to produce enjoyable daily communication with the robot. The proposed method not only controls the robot’s speech, but also controls the game development by manipulating the game characters. The proposed robot enhances a user’s enjoyment by adjusting the level of game manipulation to produce a competitive match and by speaking according to the game scene. A robot that produces speech that makes a player feel positive emotions is called a friendly robot. In order to examine the effects of these robots on a user’s enjoyment, we conducted an experiment with 30 subjects in which they were subjectively evaluated by questionnaires in two types of matches: with a friendly robot and alone. The results showed that robots were more enjoyable than playing alone.

Previous research has shown that many factors in a virtual environment influence the way, how we perceive interaction with humanoid agents. It is still unknown if visual stereotypes also translate into human-robot interaction scenarios. To investigate this relationship, we conducted a two-part user study with 26 participants using the Stereotype Content Model (SCM) to identify different stereotypical avatars and visual features in terms of their effect on the perception of robot touch. We found that humanoid avatars which resemble the robot and are perceived as warm in the SCM are more liked and preferred than cold avatars. We also found different effects of visual appearance on touch perception. This suggests that a humanoid-stereotpyical appearance of a touch-care robot in virtual reality can change the overall perception of the interaction. We propose the use of stereotypical warm avatars as described in the SCM for humanity-based applications.

In this study, we analyzed the human "gentle stroke with speech behavior" for comfortable motion by implementing the model into a robot. The results of 19 subjects’ human analysis showed that there was a positive correlation between stroke speed and speech speed. The faster the speech speed, the larger the variance of each feature. Moreover, it was also estimated regardless of the length of each sentence, people tend to fold their hands at "breaks in sentences" or "points just before breaks in sentences". Based on this human-modeled knowledge, we evaluated a human comfort experiment in 36 subjects using a robot arm to reproduce the "stroke and speech" behavior at different speeds. Results of the questionnaire showed that subjects felt more comfortable when stroke speed and speech speed were positively correlated. This result suggests that people consciously adjust their stroking and speaking styles when they perform "comfortable stroke with speech behavior".

Passengers in autonomous vehicles enjoy the comfort of being free from driving tasks, but they inevitably experience anxiety caused by autonomous vehicle stress (AVS). AVS encompasses vehicle behavior stress due to unpredictable acceleration, and external environmental stress due to potential collisions. Past research has explored approaches to improve passengers’ comfort through behavior control and information presentation. However, methods that utilize stressful vehicle behavior in Extended Reality (XR) entertainment to distract from AVS-related anxiety are limited. Hence, the goal of this study was to maximize passenger comfort in automated vehicles. To achieve this goal, we implemented an XR entertainment system that utilizes vehicle behavior and evaluated its effect on reducing anxiety. In this study, we proposed “Hype D-Live”, an XR live music system designed to reduce anxiety by providing multimodal visual, auditory, force, and vestibular stimuli using a hemispherical display and motion platform mounted on a vehicle. We developed system functions to adjust the force and vestibular senses according to the excitement level of the music and the direction of stressful acceleration and to reproduce moshing, a characteristic behavior at live music venues. However, we hypothesized that passengers might not fully enjoy the entertainment and could experience anxiety if the video content makes them aware of the external environment. Therefore, we conducted an experiment with a within-participant design, involving 24 participants (14 males and 10 females), comparing 3 types of video content for XR entertainment inside the autonomous vehicle: a real external environment, a virtual simulation of the external environment, and a virtual live music venue. The Wilcoxon signed rank test with the Bonferroni correction after the Friedman test revealed that, without the moshing function, the virtual live music venue video significantly enhanced enjoyment and reduced anxiety, compared to the real one.

Abstract Abnormal behaviour can be an indicator for a medical condition in older adults. Our novel unsupervised statistical concept drift detection approach uses variational autoencoders for estimating the parameters for a statistical hypothesis test for abnormal days. As feature, the Kullback–Leibler divergence of activity probability maps derived from power and motion sensors were used. We showed the general feasibility (min. F 1 -Score of 91 %) on an artificial dataset of four concept drift types. Then we applied our new method to our real–world dataset collected from the homes of 20 (pre–)frail older adults (avg. age 84.75 y). Our method was able to find abnormal days when a participant suffered from severe medical condition.

Recent augmented reality (AR) advancements have enabled the development of effective training systems, especially in the medical, rehabilitation, and industrial fields. However, it is unclear from the literature what the intrinsic value of AR to training is and how it differs across multiple application fields. In this work, we gathered and reviewed the prototypes and applications geared towards training the intended user's knowledge, skills, and abilities. Specifically, from IEEE Xplore plus other digital libraries, we collected 64 research papers present in high-impact publications about augmented reality training systems (ARTS). All 64 papers were then categorized according to the training method used, and each paper's evaluations were identified by validity. The summary of the results shows trends in the training methods and evaluations that incorporate ARTS in each field. The narrative synthesis illustrates the different implementations of AR for each of the training methods. In addition, examples of the different evaluation types of the current ARTS are described for each of the aforementioned training methods. We also investigated the different training strategies used by the prevailing ARTS. The insights gleaned from this review can suggest standards for designing ARTS regarding training strategy, and recommendations are provided for the implementation and evaluation of future ARTS.

One of the stress factors for passengers in an autonomous vehicle is the stress from vehicle behavior caused by unpredictable acceleration, deceleration, and route changes. Although there are researches on stress reduction methods using behavior control and information presentation to realize a comfortable autonomous vehicle, there are still few methods for stress reduction and dissipation through application to XR entertainment. In this work, we propose "Hype Dlive", a system that utilizes vehicle behavior as XR live music performance, and aims to improve passenger comfort by utilizing vehicle behavior as tactile and vestibular stimuli to generate a sense of presence and excitement.

Nowadays, decreasing opportunities for people to have daily conversations due to the increase in the number of withdrawn young people and households living alone. The lack of daily conversation has been pointed out as a risk that can lead to mental problems such as depression, and serious health problems such as dementia for the elderly. Efforts to encourage daily communication by having communication robots that act as talking partners is attracting attention to solve those problems. One of the challenges of communication robots is the difficulty of maintaining users' motivation to continue using robots. In this study, we propose the communication robot that plays a video game together with a user as an approach to keep the user's motivation high to use the robot. The proposed method not only controls the dialogue content of the robot but also controls the video game situation by manipulating the video game character. In this system, we aim to create an atmosphere where users can enjoy playing video games with the robot together, and the proposed game communication robot can keep their motivation high to use the robot.

Projector-camera systems have long been used in measuring three-dimensional shapes. Most projector-camera systems can only be used in dark rooms because frame-based cameras are not robust against strong ambient light and are difficult to obtain correspondence to the image pixels of the projector. Recently, event cameras, which can detect the direction of luminance change, have received attention in the field of computer vision. When considering the many advantages of event cameras, this study focuses on their wide dynamic range (120 vs. 40 dB of a frame-based camera) and their ability to detect fast luminance changes. Our objective is to realize a projector-camera system that combines the event camera with a projector under the strong ambient light. Specifically, this study proposes a new structured light that combines different frequencies of flickers to acquire the correspondence between the image pixels of the event camera and the projector. This method does not rely on the co-axial frame-based measurement and synchronization mechanism between projector and camera and is thus applicable to most general event cameras. Experiments confirm that the proposed method obtains the correspondence robustly with reasonable accuracy in a bright room (up to 2,600 lux) under general indoor lighting and additional light projection.

Humanitude is a multimodal communication care method that utilizes seeing, touching, and speaking. Moreover, a touch-care method is well known as an effective care method mainly focus on touch motion. These kinds of care techniques are effective in practical situations, however, it is difficult to provide such care therapy to all patients due to the lack of human resources. To address this problem, researchers try to develop a touch-care robot that can provide touch-care automatically. Conventional research of touch-care robot mainly focuses on the movement of stroke or the speech that only considers the impression of the contents of speech but not prosodic information. Therefore, in this research, we focus on the speech rate in the prosodic information with stroke motion. In this work, we investigate the effects of speech rate on the prosodic information and evaluate the relationship between stroke pace and speech rate to improve human comfort. We conducted a user study with 6 participants around 20 years old males. As a result of the list of the questionnaire suggests a correlation between stroke pace and speech rate that provides comfort.

Personal mobility devices such as wheelchairs, bicycles, and compact cars are used in daily life and to runs on sidewalks. However, there are several factors that may lead to discomfort rides, such as steps, slopes, and crowded sidewalks for a passenger. This paper proposes a system that detects the factors using smartphone attached to the personal mobility device and generates a discomfort-ride map including the environment and human factors and their rating on sidewalks. In the experiment, steps (static danger factor) and the dynamic moving obstacles like human, bicycle, and car (dynamic danger factors) were estimated with attached smartphone's acceleration sensor and camera. After the process of collecting these data, verification for generating the hazard map system based on these data. Moreover, the versatility of using the proposed hazard map system for different personal mobility devices was tested with the electric wheelchair and the bicycle.

This paper presents an approach for motion sickness reduction while riding an autonomous vehicle. It proposes the Diminished Reality (DR) method for an acceleration stimulus to reduce motion sickness for the autonomous vehicle. One of the main causes of motion sickness is a repeated acceleration. In order to diminish the acceleration stimulus in the autonomous vehicle, vection illusion is used to induce the user to make a preliminary movement against the real acceleration. The Balance Wii Board is used to measure participant's movement of the center of gravity to verify the effectiveness of the method with vection. The experimental result of 9 participants shows that the proposed method of using vection could reduce acceleration stimulus compared with the conventional method.