Hydraulic excavators are widely used in construction work owing to their versatility. However, the general operation of these excavators is complex and novice operators require extensive training to operate them. In this study, we propose a virtual reality (VR)-based training system with three types of visual augmentations using pre-recorded expert operations to support the skill acquirement for handling a hydraulic excavator. To evaluate the effectiveness of the proposed visual augmentations in terms of skill improvement, we compared the scores of the trainees before and after training including combinations of visual augmentations. The results indicated that the display of the lever movement significantly improved the trajectory of the bucket tip, while ghost animation and slow motion did not show significant effects. Furthermore, by showing the lever input and excavator movement of the expert in slow motion, the task completion time increased because of the aftereffect. Our findings not only provide a design guideline for VR-based excavator operation training but can also be applied to augmented reality (AR)/mixed reality (MR) support systems for supporting practical excavator operations.

In this study, we focused on one-to-many remote collaboration which requires more mental resources from the remote instructor than the case of one-to-one since it is "multitasking". The main contribution of our study is that we assessed instructor’s capacity in one-to-many AR remote instruction giving both subjectively and objectively. We compared the remote instructor’s workload while interacting with a different number of local workers, assuming tasks at an industrial site. The results showed that the instructors perceived stronger workload and the communication quality became lower when interacting with multiple local workers. Based on the results, we discussed how to support the remote instructor in a one-to-many AR remote collaboration.

The emergence of new and inexpensive virtual reality (VR) technology has made it relatively familiar to most users. We can create virtual 3D objects and paint on them in a VR spaces. Many of the VR controllers used for such operations provide haptic feedbacks by vibration when the user touches the virtual objects. In the real world, we can perceive not only the touch sensation but also the reaction force when touching and stroking the object's surface. However, it is impossible to provide a reaction force only with the vibration feedback. That is, there is a sensory gap between the VR space and the real world. The gap makes it difficult to work in the VR space in a manner similar to that in the real world. In this study, we focused on providing the reaction force from the virtual object to the user, and proposed a device that could provide the force feedback to the user's arm without connecting the device to large equipment.

We developed Sports Support System that augments the perception of beginner players and supports situation awareness to motivate beginners in multiplayer sports through visual and haptic feedback. Our system provides positional relationship of the opponents using visual feedback. In addition, the position of the opponent beyond the field of view was provided using haptic feedback. An experiment of pass interception as used to compare the visual and haptic feedback. The results confirmed the effectiveness and characteristics of these feedback processes in a multiplayer ball game.

In this paper, we compared three different types of avatar design (“Body”, “Hand + Arm”, and “Hand only”) for the augmented reality remote instruction system in terms of usability. The result showed that the usability of the remote instruction system with full body avatar has a higher usability. In addition, participants felt more easily to track the full body avatar than the avatar with hand only. However, concerning the understandability of the instruction, there was no difference between three designs.

With respect to collaborative physical tasks, gaze and gestures play significant roles when referring to physical objects. In video-mediated communication, however, such nonverbal cues become "ineffectual" when they are presented via a 2D monitor, making video-mediated collaborative physical tasks inefficient. This study focuses on gaze cues to support remote collaborative physical tasks and uses an eye-shaped display, "ThirdEye," a simple add-on display that represents a remote participant's gaze direction. ThirdEye is expected to be especially effective when used with mobile terminals. We investigated whether the ThirdEye's gaze shift is effective in leading a local observer's attention toward objects in the local environment, even when ThirdEye is presented with the actual face image of a remote person. Experimental results show that ThirdEye can lead the local participant's attention to intended objects faster than without ThirdEye.

Virtual and mixed realities make it possible to view and interact with virtual objects in 3D space. However, where to position menus in 3D space and how to interact with them are often problems. Existing studies developed methods of displaying a menu on the hand or arm. In this study, we proposed a menu system that appears at various body parts. By placing the menu on the body, it enables the user to operate the menus comfortably through kinesthesia, and perceive tactile feedback. Furthermore, displaying the menu not only in the hands and arms but also in the upper legs and the abdomen, the menu display area can be expanded. In this study, we developed a modeling application and introduced a proposed menu design for that application.

We propose to artificially manipulate participants' vocal cues, amplitude and frequency, in real time to adjust their dominance and persuasiveness during audio conferences. We implemented a prototype system and conducted two experiments. The first experiment investigated the effect of vocal cue manipulation on the perception of dominance. The results showed that participants perceived higher dominance while listening to a voice with a high amplitude and low frequency. The second experiment investigated the effect of vocal cue manipulation on persuasiveness. The results indicated that a person with a low amplitude and low frequency voice had greater persuasiveness in conversations with biased dominance, while there was no statistically significant difference in persuasiveness in conversations with unbiased dominance.

A long-standing challenge in video-mediated communication systems is to represent a remote participant's gaze direction in local environments correctly. To address this issue, we developed ThirdEye, an add-on eye-display for a video communication system. This display is made from an artificial ulexite (TV rock) that is cut into a hemispherical shape, enabling light from the bottom surface to be projected onto the hemisphere surface. By drawing an appropriate ellipse on an LCD and placing ThirdEye over it, this system simulates an eyeball. Our experiment proved that an observer could perceive a remote Looker's gaze direction more precisely when the gaze was presented using ThirdEye compared to the case in which the gaze was presented using the Looker's face on a flat display.

In this paper, we propose a system that can carve virtual objects in Mixed Reality (MR) space. The procedures of real-world carving include sculpting a rough outline, shaping sections, and carving patterns onto an object's surface. Of these, we focus on the procedure for carving patterns. Users of our system stroke a real object directly using ChiselDevice, and the surfaces of 3D virtual objects superimposed on the real object are "carved." This paper describes the design and development of the ChiselDevice and the MR carving system and the findings of users' experiences of our system.

In this study, we used a humanoid robot as a telepresence robot and compared with the basic telepresence robot which can only rotate the display in order to reveal the effect of embodiment. We also investigated the effect caused by changing the body size of the humanoid robot by using two different size of robots. Our experimental results revealed that the embodiment increases the remote person's social telepresence, familiarity, and directivity. The comparison between small and big humanoid robots showed no difference and both of them were effective.

In order to fill the gap between various levels of ball game players, we propose the Sports Support System that can reduce the difference among the skills of various players by augmenting the traditional ball game. We implemented a system that visualizes the trajectory and velocity of the ball for soccer games, as one example of ball games. This visualization and the enhancement of the player positions in the field can help beginner players recognize what they should do next and improve their playing skills. This paper introduces the proposed system and discusses, through a user study, its effectiveness for improving reaction speed and enhancing the pleasure of playing sports. Based on the experiment results, the proposed system contributes to reaction time improvements in passing and receiving the ball, i.e., one of the most important skills in playing soccer.

Most 3-D modeling software are difficult for beginners to learn. The operations are often complicated, and the user is required to have prior mathematical knowledge. Therefore, we developed a simple modeling system using ToolDevice to simplify such operations. ToolDevice consists of a set of interaction devices, which use metaphors of real-life hand tools to help users recognize each device's unique functions. Using TweezersDevice, KnifeDevice, and HammerDevice, we developed a mixed reality (MR) 3-D modeling system that imitates real-life woodworking. In the system, TweezersDevice is used to pick up and move objects, while KnifeDevice and HammerDevice are, respectively, used to cut and join virtual objects represented as wood materials. In this study, we describe the motivation for developing the system, the available interactions, and the procedures for creating 3-D models in the system. We also present the results of a user study in which we compare user performance in our system and a common 3-D modeling software. Finally, we discuss the contributions and limitations of this study and future work.

We report an experiment related to perceiving (virtual) objects in the vicinity of (real) surfaces when using stereoscopic augmented reality displays. In particular, our goal was to explore the effect of various visual surface features on both perception of object location and perception of surface transparency. Surface features were manipulated using random dot patterns on a simulated real object surface, by manipulating dot size, dot density, and whether or not objects placed behind the surface were partially occluded by the surface.

We report a two part experiment related to perceiving (virtual) objects in the vicinity of (real) surfaces when using stereoscopic augmented reality displays. In particular, our goal was to explore the effect of various visual surface features on both perception of object location and perception of surface transparency. Surface features were manipulated using random dot patterns on a simulated real object surface, by manipulating dot size, dot density, and whether or not objects placed behind the surface were partially occluded by the surface.

Today's technology enables users to manipulate complex, multi-part 3D virtual objects such as industrial products, structures designed by CAD, and models of the human body in large 3D space. In general modeling software, parts of such a complex 3D virtual objects are grouped and manipulated together, but not individually, for efficient operation. Therefore, a separating operation is necessary when the user wants to observe or manipulate only a part of a complex object. We have developed a system that realizes gesture-based separation and observation of a group of parts within complex virtual objects in 3D space. One of the practical applications of our system is in training, such as learning the structure of the human body or industrial products. In our system, users can separate a group of parts freely by pulling and cutting a virtual rubber band. The widths of these virtual rubber bands are relative to the connection strength between each part. This allows the user to easily understand relationships between parts. Additionally, this feedback provides a comfortable operational familiarity.

Many digital painting systems have been proposed and their quality is improving. In these systems, graphics tablets are widely used as input devices. However, because of its rigid nib and indirect manipulation, the operational feeling of a graphics tablet is different from that of real paint brush. We solved this problem by developing the MR-based Artistic Interactive (MAI) Painting Brush, which imitates a real paint brush, and constructed a mixed reality (MR) painting system that enables direct painting on physical objects in the real world.