Gesture-based locomotion enhances immersion in virtual reality (VR), with seated motion being crucial for accessibility and prolonged use. However, existing techniques often apply uniform gesture-to-walking speed mappings, ignoring the fact that different gestures involve varying levels of physical effort and subjective impressions. This mismatch can degrade the user experience. This study investigates how three seated gestures with different physical loads—Tap-in-Place (TIP), Swing-in-Place (SIP), and Grip-in-Place (GIP)—influence users’ expected walking speed. While the evaluations revealed unique experiential trade-offs for each gesture, our primary finding is a consistent perceptual pattern in the expectation of walking speed: Users expected to walk fastest with SIP, followed by GIP, then TIP (SIP > GIP > TIP). These results demonstrate that a one-size-fits-all approach is insufficient and provide empirical recommendations for designing more intuitive seated VR locomotion systems that align walking speed with user perception.

Users in a prolonged experience of virtual reality adopt a sitting position according to their task, as they do in the real world. However, inconsistencies in the haptic feedback from a chair they sit on in the real world and that which is expected in the virtual world decrease the feeling of presence. We aimed to change the perceived haptic features of a chair by shifting the position and angle of the users' viewpoints in the virtual reality environment. The targeted features in this study were seat softness and backrest flexibility. To enhance the seat softness, we shifted the virtual viewpoint using an exponential formula soon after a user's bottom contacted the seat surface. The flexibility of the backrest was manipulated by moving the viewpoint, which followed the tilt of the virtual backrest. These shifts make users feel as if their body moves along with the viewpoint; as a result, they would perceive pseudo-softness or flexibility consistently with the body movement. Based on subjective evaluations, we confirmed that the participants perceived the seat as being softer and the backrest as being more flexible than the actual ones. These results demonstrated that only shifting the viewpoint could change the participants' perceptions of the haptic features of their seats, although significant changes created strong discomfort.

The emergence of new and inexpensive virtual reality (VR) technology has made it relatively familiar to most users. We can create virtual 3D objects and paint on them in a VR spaces. Many of the VR controllers used for such operations provide haptic feedbacks by vibration when the user touches the virtual objects. In the real world, we can perceive not only the touch sensation but also the reaction force when touching and stroking the object's surface. However, it is impossible to provide a reaction force only with the vibration feedback. That is, there is a sensory gap between the VR space and the real world. The gap makes it difficult to work in the VR space in a manner similar to that in the real world. In this study, we focused on providing the reaction force from the virtual object to the user, and proposed a device that could provide the force feedback to the user's arm without connecting the device to large equipment.

In mixed reality using video see-through displays, various optical effects can be simulated for representing a transparent object. However, consumer-available OST-HMDs can not reproduce some of their effects correctly because of its mechanism. There has been little discussion concerning to what extent such transparent objects can be represented realistically. In this demonstration, we will show how effective each for combination of existing methods is.

Virtual shadows provide important cues to determine the positional relationship between virtual objects and a real scene. However, it' s difficult to render shadows in the real scene on optical see-through head-mounted displays without occlusion-capability. In the previous work, we cast virtual shadows not with physical light attenuation but with brightness induction caused by virtual objects, referred to as shadow inducers, which surround the shadow area to gradually amplify the intensity of the real scene pattern [4]. However, because the shadow inducer was prepared beforehand, the shape of the shadow is constant, the real scene shadowing is limited to a flat surface, and a large of viewpoint change is impossible. In this demonstration, we propose a method to generate shadow inducers in real time that can change the shape of virtual objects and the viewpoint of users. In this method, depending on the appearance of shadows, the surrounding luminance is gradually amplified with the difference of gaussian (DOG) representing characteristics of human vision. Users can observe shadows of moving and deforming virtual objects on a real tabletop and other non-planar objects.

This paper proposes a novel method for casting virtual shadows on real surfaces on an optical see-through head-mounted display without any extra physical filter devices. Instead, the method presents shadows as results of brightness induction. To produce brightness induction, we place a texture of the real scene with a certain transparency around the shadow area to amplify the luminance of the surrounding area. To make this amplification unnoticeable, the transparency of the surrounding region is gradually increased as the distance from the shadow region. In the experiment with 23 participants, we confirmed that users tend to perceive the shadow region is darker than a non-shadow area under the conditions where a circular virtual shadow is placed on a flat surface.

Virtual and mixed realities make it possible to view and interact with virtual objects in 3D space. However, where to position menus in 3D space and how to interact with them are often problems. Existing studies developed methods of displaying a menu on the hand or arm. In this study, we proposed a menu system that appears at various body parts. By placing the menu on the body, it enables the user to operate the menus comfortably through kinesthesia, and perceive tactile feedback. Furthermore, displaying the menu not only in the hands and arms but also in the upper legs and the abdomen, the menu display area can be expanded. In this study, we developed a modeling application and introduced a proposed menu design for that application.

We can use augmented reality (AR) and diminished reality (DR) in combination, in practice. However, to the best of our knowledge, there is no research on the validation of the cross-modal effects in AR and DR. Our research interest here is to investigate how this continuous visual changes between AR and DR would change our weight sensation of an object. In this paper, we built a system that can continuously extend and reduce the amount of visual entity of real objects using AR and DR renderings to confirm that users can perceive things heavier and lighter than they actually are in the same manner as SWI. Different from the existing research where either AR or DR visual effects were used, we validated one of cross-modal effects in the context of both continuous AR and DR visuo-haptic. Regarding the weight sensation, we found that such cross-modal effect can be approximated with a continuous linear relationship between the weight and length of real objects. Our experimental results suggested that the weight sensation is closely related to the positions of the center of gravity (CoG) and perceived CoG positions lie within the object's entity under the examined conditions.

The R-V Dynamics illusion is a phenomenon where weight perception is changed by superimposing a CG case with a movable portion (CG) onto a real object using mixed reality technology. In previous studies, it has been confirmed that weight perception is affected by the size/volume of the CG, and a virtual collision sound between the case and the movable portion could also be a cause of this illusionary phenomenon. However, in previous studies, only one virtual collision sound is applied. Therefore, in this study, we consider the influence of the physical characteristics of virtual collision sound such as the size and weight of the movable object in the phenomenon. As a result, it was confirmed that the weight perception changes according to the virtual collision sound, and participants lightly perceived the real object when a virtual collision sound is played with a smaller and lighter object.

In this paper, we propose a system that can carve virtual objects in Mixed Reality (MR) space. The procedures of real-world carving include sculpting a rough outline, shaping sections, and carving patterns onto an object's surface. Of these, we focus on the procedure for carving patterns. Users of our system stroke a real object directly using ChiselDevice, and the surfaces of 3D virtual objects superimposed on the real object are "carved." This paper describes the design and development of the ChiselDevice and the MR carving system and the findings of users' experiences of our system.

In Mixed-Realty (MR) space, the visual appearance (shape, texture, etc.) of a real object can be changed by superimposing a virtual object on it. Therefore, by creating systematic differences between visual and haptic perceptions using MR technology, we can analyze their influence on temperature perception. In our research group, we defined the changes in the visual information of a real object in MR space as “MR visual stimulation” and examine the influence of the haptic sense using MR visual stimulation [1]. For our research, we focused on the temperature perception of the skin. In the first step, we verified the influence presenting MR visual stimulation has on the perceived position of the temperature perception. In the experiment, we presented MR visual stimulation and temperature stimulation in different positions. We confirmed the influence this difference has on the temperature perceived position. Our results demonstrate that temperature perception is strongly affected by visual stimulation.

Most 3-D modeling software are difficult for beginners to learn. The operations are often complicated, and the user is required to have prior mathematical knowledge. Therefore, we developed a simple modeling system using ToolDevice to simplify such operations. ToolDevice consists of a set of interaction devices, which use metaphors of real-life hand tools to help users recognize each device's unique functions. Using TweezersDevice, KnifeDevice, and HammerDevice, we developed a mixed reality (MR) 3-D modeling system that imitates real-life woodworking. In the system, TweezersDevice is used to pick up and move objects, while KnifeDevice and HammerDevice are, respectively, used to cut and join virtual objects represented as wood materials. In this study, we describe the motivation for developing the system, the available interactions, and the procedures for creating 3-D models in the system. We also present the results of a user study in which we compare user performance in our system and a common 3-D modeling software. Finally, we discuss the contributions and limitations of this study and future work.

When humans sense the hardness of real objects, their perception is known to be influenced by not only tactile information but also visual information. In a mixed-reality (MR) environment, the appearance of touchable objects can be changed by superimposing a computer-generated image onto them (MR visual stimulation). In our previous study, we first superimposed computer-generated images (CGI) on real objects, and then, applied extreme deformation to the superimposed images while the participants pressed the real objects with their fingers. The results showed that humans sense hardness differently while receiving MR visual stimulation; we named this psychophysical effect the dent-softness illusion. As the next step, in this study, we verify whether humans perceive hardness differently when pressing a real object with and without superimposed animation and whether the dent-softness illusion occurs in different parts of the body such as the finger, palm, and arm.

We focus on a challenging trial of a Look-Change mixed reality (MR) space. Look refers to perceived ambience of an image created by illumination and color tone. Regarding the Look-Change, we have been working on a relighting technique which allows MR space to have additional virtual illumination. Furthermore, we demonstrate a photorealistic approach to create photorealistic virtual objects with free-form polarized reflectometry which requires no specialized hardware and is portable, thus making it easy apply to MR.

Mixed reality (MR) is a technology that merges real and virtual worlds in real time. In MR environments, visual appearance of a real object can be changed by superimposing a virtual object on it. In this study, we focus on the center-of-gravity (COG) and verify the influence of MR visual stimulation on the COG in MR environments. This paper describes the systematic experiments performed to study the influence. The results obtained are interesting: (1) the presence of COG can be changed by MR visual stimulation; (2) although COG differs in vision and force, the presence of COG can be represented by MR visual stimulation under certain conditions; (3) COG perception can also be changed by varying the mass of the real object. We named this psychophysical influence the “Shape-COG Illusion.”

Today's technology enables users to manipulate complex, multi-part 3D virtual objects such as industrial products, structures designed by CAD, and models of the human body in large 3D space. In general modeling software, parts of such a complex 3D virtual objects are grouped and manipulated together, but not individually, for efficient operation. Therefore, a separating operation is necessary when the user wants to observe or manipulate only a part of a complex object. We have developed a system that realizes gesture-based separation and observation of a group of parts within complex virtual objects in 3D space. One of the practical applications of our system is in training, such as learning the structure of the human body or industrial products. In our system, users can separate a group of parts freely by pulling and cutting a virtual rubber band. The widths of these virtual rubber bands are relative to the connection strength between each part. This allows the user to easily understand relationships between parts. Additionally, this feedback provides a comfortable operational familiarity.

PreViz is a computer graphics movie that represents desired scenes in preproduction and is useful for sharing the director's imagination among crews. Therefore, film industry looks upon PreViz as one of the most important processes in filmmaking as the processes become complicated [The Previs Society 2011]. Stereoscopic 3D (S3D) movie production not only changed ways of expressiveness but also changed ways of production and tools. Consequently, it requires more careful planning and PreViz is still important. This paper describes a prototype of S3D MR-PreViz system for HD S3D PreViz shooting using mixed reality (MR) technologies in order to facilitate PreViz making in S3D. This system is designed as an extension of MR-PreViz for classic filmmaking [Ichikari et al. 2010].

In a mixed-reality (MR) environment, the appearance of touchable objects can be changed by superimposing a computer-generated image (CGI) onto them (MR visual stimulation). At the same time, when humans sense the hardness of real objects, it is known that their perception is influenced not only by tactile information but also by visual information. In this paper, we studied the psychophysical influence on the sense of hardness by using a real object that has a CGI superimposed on it. In this experiment, we deform in an extreme way the CGI animation on the real object, while the subject pushes the real object using his/her finger. The results of the experiments found that human subjects sensed different hardnesses by emphasizing the dent deformation of the CGI animation.

In the study of AR/MR field, tracking and geometric registration methods are very important topics that are actively discussed. Especially, the study on tracking is flourishing and many algorithms are being proposed every year. With this trend in mind, we, the TrakMark WG, had proposed benchmark test schemes for geometric registration and tracking in AR/MR at ISMAR 2009 [1]. This paper is an intermediate report of the TrakMark WG, which describes its activities and the first proposal on benchmarking image sequences.

In this paper, we present various novel tangible devices suitable for interactions in a mixed reality (MR) environment. They are aimed at making the best use of the features of MR, which allows users to touch or handle both virtual and physical objects. Furthermore, we consider usability and intuitiveness as important characteristics of the interface, and thus designed our devices to imitate traditional tools and help users understand their use.

Many digital painting systems have been proposed and their quality is improving. In these systems, graphics tablets are widely used as input devices. However, because of its rigid nib and indirect manipulation, the operational feeling of a graphics tablet is different from that of real paint brush. We solved this problem by developing the MR-based Artistic Interactive (MAI) Painting Brush, which imitates a real paint brush, and constructed a mixed reality (MR) painting system that enables direct painting on physical objects in the real world.

This paper describes the constructing method of consistent action scene data of CG charactors. The proposed method realizes to construct multiple characters' action scenes by merging one actor's action sequences with keeping time and space consistencies between multiple individually recorded or manually-designed action sequences semi-automatically. This technique is planned as a part of "Mixed Reality Pre-visualizing As Pre-Production Tool in Film-making; MR-PreViz" which can streamline the filmmaking process by overlaying CG action data on the real background image in real time.

This paper describes the influence of visual stimulation on the tactual sense in a mixed-reality environment; i.e., how a tactual impression of a real object is affected by seeing a superimposed image of a different type of material. If the behavior and the extent of such an influence, a sort of illusion, are investigated in detail, the objects composed of a limited variety of materials can be perceived differently. This would be useful in the field of digital engineering. Therefore, we performed various experiments systematically.

There have been many implementations of virtual reality, using audio and visual senses. However, implementations of mixed reality (MR) have thus far only dealt with the visual sense. We have developed an MR system that merges real and virtual worlds in both the audio and visual senses, wherein the geometric consistency of the audio sense was fully coordinated with the visual sense. We tried two approaches for merging real and virtual worlds in the audio sense, using open-air and closed-air headphones.

This paper describes a novel image-based geometric registration method using visually elegant and robust semi-fiducial markers in mixed reality (MR). Most traditional visual markers stand out against the background, i.e., they can be recognized and identified easily. Here, we try to construct new visual markers for MR registration, which achieve a good balance between visual elegance and robustness. As the first step, we propose "two-tone colored markers". These markers have the following characteristics: 1) color similar to that of the background object, 2) placed at the corners of real objects in an inconspicuous manner. This paper describes the registration method using two- tone colored markers and a few experiments.

In this paper, we introduce the outline of "The MR-PreViz Project" performed in Japan. In the pre-production process of filmmaking, Pre Viz, pre-visualizing the desired scene by CGI, is used as a new technique. As its advanced approach, we propose MR-PreViz that utilized mixed reality technology in current PreViz. MR-PreViz makes it possible to merge the real background and the computer-generated humans and creatures in open set or at outdoor location. The user can consider the camerawork and camera blocking efficiently by using MR-PreViz. This paper introduces the outline of the MR-PreViz project, the design of hardware configuration, camera-work authoring method and the results of prototyping.

We propose a route planning algorithm for agent navigation under linguistic instructions of a route, aiming at man-agent flexible communication by a natural language. Instructions given to the agent describe a route in terms of landmarks like intersections, for example, "turn right at the second intersection". The contextual information in instructions of a route provides some constraints on the interpretation of landmarks and their geometric relationships. This information helps to reduce ambiguities in landmark recognition.