In this paper, we analyze the subjective feelings about the body of the operator of a telexistence system. We investigate whether a mirror reflection and self-touch affect body ownership and agency for a surrogate robot avatar in a virtual reality experiment. Results showed that the presence of tactile sensations synchronized with the view of self-touch events enhanced mirror self-recognition.

We have developed a classification method of tactile feeling using a stacked autoencoder-based neural network on haptic primary colors. The haptic primary colors principle is a concept of decomposing the human sensation of tactile feeling into force, vibration, and temperature. Images were obtained from variation in the frequency of the time series of the tactile feeling obtained when tracing a surface of an object, features were extracted by employing a stacked autoencoder using a neural network with two hidden layers, and supervised learning was conducted. We confirmed that the tactile feeling for three different surface materials can be classified with an accuracy of 82.0 [%].

The information we acquire in real life gives us a holistic experience that fully incorporates a variety of sensations and bodily motions such as seeing, hearing, speaking, touching, smelling, tasting, and moving. However, the sensory modalities that can be transmitted in our information space are usually limited to visual and auditory ones. Haptic information is rarely used in the information space in our daily lives except in the case of warnings or alerts such as cellphone vibrations. Embodied media such as virtual reality and telexistence should provide holistic sensations, i.e., integrating visual, auditory, haptic, palatal, olfactory, and kinesthetic sensations, such that human users feel that they are present in a computer-generated virtual information space or a remote space having an alternate presence in the environment. Haptics plays an important role in embodied media because it provides both proprioception and cutaneous sensations; it lets users feel like they are touching distant people and objects and also lets them “touch” artificial objects as they see them. In this keynote, an embodied media, which extends human experiences, is overviewed and our research on an embodied media that is both visible and tangible based on our proposed theory of haptic primary colors is introduced. The embodied media would enable telecommunication, tele-experience, and pseudo-experience providing sensations such that the user would feel like working in a natural environment. It would also enable humans to engage in creative activities such as design and creation as though they were in the real environment. We have succeeded in transmitting fine haptic sensations, such as material texture and temperature, from an avatar robot’s fingers to a human user’s fingers. The avatar robot is a telexistence anthropomorphic robot, called TELESAR V, with a body and limbs with 53 degrees of freedom. This robot can transmit not only visual and auditory sensations of presence to human users but also realistic haptic sensations. Our other inventions include RePro3D, which is a full-parallax autostereoscopic 3D (three-dimensional) display with haptic feedback using RPT (retroreflective projection technology); TECHTILE Toolkit, which is a prototyping tool for the design and improvement of haptic media; and HaptoMIRAGE, which is an 180°-field-of-view autostereoscopic 3D display using ARIA (active-shuttered real image autostereoscopy) that can be used by three users simultaneously.

We propose an indirect-vision, video-see-through augmented reality (AR) cockpit that uses telexistence technology to provide an AR enriched, virtually transparent view of the surroundings through monitors instead of windows. Such a virtual view has the potential to enhance driving performance and experience above conventional glass as well as head-up display equipped cockpits by combining AR overlays with images obtained by future image sensors that are superior to human eyes. As a proof of concept, we replaced the front windshield of an experimental car by a large stereoscopic monitor. A robotic stereo camera pair that mimics the driver's head motions provides stereoscopic images with seamless motion parallax to the monitor. Initial driving tests at moderate speeds on roads within our research facility confirmed the illusion of transparency. We will conduct human factors evaluations after implementing AR functions in order to show whether it is possible to achieve an overall benefit over conventional cockpits in spite of possible conceptual issues like latency, shift of viewpoint and short distance between driver and display.

Rubber Hand Illusion (RHI) is known to provide a proprioceptive drift toward the direction where the rubber hand is tapped. On the other hand, Out-of-Body Experiences (OBEs) induce an entire body shift from a third-person Point of View (PoV). These experiences are known to trigger external passive tactile cues. Telexistence allows an entire body shift from a first-person PoV, compared to RHI and OBE. However, the precise origin of the entire localization is not revealed when we actively provide the internal voluntary tactile cue. Therefore, in this paper, we focus on the effects of tactile cues to determine self-localization during the action of self-tapping. Participants viewed and tapped their own body on the back, and evaluated where on their body they perceived the tapping. The experiments show that the localized position tends to shift forward when an individual is tapped by a third-person; in contrast, the localized position tends to shift backward when an individual is tapping his own back. The subjective report indicates that participants perceived themselves as leaving their own bodies, or that someone appeared in front of them suddenly. Thus, we consider that self-produced tactile cue induces telexistence experience.

We present Reality Jockey, a system that confuses the participant's perception of the reality by mixing in a recorded past-reality. The participant will be immersed in a spatialized 3D sound environment that is a mix of sounds from the reality and from the past. The sound environment from the past is augmented with haptic feedback in cross-modality. The haptic feedback is associated with certain sounds such as the vibration in the table when stuff is placed on the table to make the illusion of it happening in live. The seamless transition between live and past creates immersive experience of past events. The blending of live and past allows interactivity. To validate our system, we conducted user studies on 1) does blending live sensations improve such experiences, and 2) how beneficial is it to provide haptic feedbacks in recorded pasts. Potential applications are suggested to illustrate the significance of Reality Jockey.

Summary form only given. Virtual reality (VR) captures the essence of reality and is effectively real. VR can provide a basis for technology that enables humans to experience events and to act in a computer-synthesized environment as if they were in a real environment, while augmented reality (AR) seamlessly integrates virtual environments into real environments to enhance the real world. Telexistence is a concept that frees humans from the restrictions of time and space, thereby allowing them to exist virtually in remote locations, as well as to interact with remote environments that may be real, computer-synthesized, or a combination of both. This keynote reviews recent advancements in VR, AR, and telexistence with a special emphasis on retroreflective projection technology (RPT), such as optical camouflage and Repro3D, a mutual telexistence system called TELESAR IV, TELESAR V for achieving haptic telexistence, and a telexistence wide-angle immersive stereoscope (TWISTER) with a fullcolor autostereoscopic display and a 360-degree field of view. Social interactions have always been an important part of human learning and experience. We now know that social interactions are critical in many knowledge and information processes. Research has shown results ranging from influences on our behavior from social networks [Aral2012] to our understanding of social belonging on health [Walton2011], as well as how conflicts and coordination play out in Wikipedia [Kittur2007]. Interestingly, social scientists have studied social interactions for many years, but it wasn't until very recently that researchers are able to study these mechanisms through the explosion of services and data available on web-based social systems.

In this paper, we propose a haptic transmission system for telexistence to improve the ability to sense the presence of remote objects. This system can transmit information about the existence and surface textures of objects in remote locations. The system consists of a conjugated haptic sensor and display. The sensor on the robot's finger detects the pressure, vibration, and temperature of a remote object, and the display provides above information on the operator's finger. Based on this information, the operator can understand what he/she is touching and whether its surface is hard or soft, cold or hot, and smooth or rough. With the use of our system, the operator can recognize the difference between objects such as silk and denim.

This paper focuses on design of a dexterous anthropomorphic robot where the operator can perceive the transferring bodily consciousness to the slave robot during a tele-operation. Accordingly, we propose a telexistence surrogate anthropomorphic robot called “TELESAR V”, which was designed and constructed by development of the following: a 52 DOF slave robot with a torso, upper limbs, hands and head to model the operator's posture on all parts of the upper body and maintain a 6 DOF accuracy in arm endpoint; a HD Head mounted display with 6 DOF point of view accuracy for wide angle stereovision; and a mechanism for sensing and reproducing fingertip haptic and thermal sensation. This paper describes the development of the TELESAR V system, where the effectiveness has been verified through functional experiments.

Telexistence technology enables a highly realistic sensation of existence in a remote place without any actual travel. The concept was originally proposed by the first author in 1980, and its feasibility has been demonstrated through the construction of alter-ego robot systems such as TELESAR & TELESAR V, which were developed under the national large scale project on “Robots in Hazardous Environments” and the “CREST Haptic Telexistence Project.” A mutual telexistence system, such as TELESAR II & IV, capable of generating the sensation of being in a remote place in local space using a combination of an alter-ego robot and retro-reflective projection technology (RPT), has been developed, and the feasibility of mutual telexistence has been demonstrated. Thirty-two years of telexistence development are historically reviewed in this jubilee video.

In this paper, we implement an ungrounded pen-shaped kinesthetic display and construct a three-dimensional (3D) haptic interaction system. The ungrounded pen-shaped kinesthetic display provides kinesthetic sensations to a user's fingers without the use of mechanical linkages. Therefore, the user can move his/her hand freely in air and interact with virtual environments with sensation of touching. We verified the ability of the device to provide forces and to represent a virtual surface for 3D input. Then we constructed a 3D haptic interaction system in that the user can touch virtual objects displayed as 3D images directly. We also applied the device to an interface for 3D modeling and constructed a 3D haptic modeling system in that the user can create touchable 3D models by sketching two-dimensional (2D) figures in air.

A thermal change on a fingertip is essential for haptic perception. We have proposed a vision-based thermal sensor using thermo-sensitive paint and a CCD camera for telexistence. The thermo-sensitive paint is employed to measure thermal information on the basis of its color, which changes according to its temperature. The proposed sensor can simulate the physical interaction between a human fingertip and an object in order to measure surface thermal information correctly. Furthermore, because the proposed sensor can be easily integrated with our vision-based force sensor, a comprehensive measurement device for measuring haptic information can be realized. In this study, we constructed a prototype of the proposed thermal sensor and experimentally confirmed that this sensor could measure surface thermal information.

Our aim is to afford a remote person the opportunity to participate virtually in some event by using a surrogate robot to communicate with local participants while moving about freely at the venue. Accordingly, we propose a mutual telexistence surrogate robot system, called TELESAR4, which was designed and constructed by development of the following: an immersive audiovisual system; an omnidirectionally mobile robot with a robot arm and hand; an omnidirectional stereo camera system (VORTEX); a head with a retroreflective screen for embodiment of the remote participant; and a retroreflective projection system for local participants. This paper describes the development of the TELESAR4 system, whose efficacy has been verified through demonstration experiments.

We propose a three-dimensional (3D) haptic modeling system that enables a user to create 3D models as though he/she is drawing pictures in mid-air and touch the created models. In the proposed system, we use our ungrounded pen-shaped kinesthetic display as the interface. The kinesthetic display device can generate kinesthetic sensations on the user's fingers. Because our system does not use mechanical linkages, the user can freely move his/her hand and feel the sensation of touching virtual objects. In the proposed system, the user can easily and intuitively create various 3D shapes by drawing closed curves in air using the device. The created shapes are generated in a physics-based simulation environment and are displayed as 3D images. Therefore, the user can touch and see the drawn shapes as though they exist in reality.

We propose a novel interface called Twinkle for interacting with an arbitrary physical surface using a handheld projector and a camera. When a user flashes a projection light on an object, the projected images react as if the user touched the object with the light. The handheld device recognizes the features of the physical environment and displays images and sounds that are generated in real-time according to the user's motion and collisions of projected images with objects. We realize this system by using several image-processing techniques and a collision detection algorithm. We also use an acceleration sensor to compensate for the image processing. In this paper, we explain the principle of interacting with a physical surface. Then, we describe the implementation of the prototype system and some application examples.

We propose a novel grip-type master hand called MeisterGRIP that measures grip force in terms of a force vector distribution. This device is expected to allow intuitive robot manipulation using vision-based haptic-sensing technology. Furthermore, it can be used for general-purpose manipulation and is tolerant to individual differences in hand size and grasping posture. We constructed MeisterGRIP and evaluated the accuracy of the measured grip force. Furthermore, we constructed and exhibited a complete robot manipulation system using MeisterGRIP to demonstrate the possibility of using MeisterGRIP as a general-purpose master hand.

We proposed a pen-shaped handheld haptic display that provides kinesthetic sensations to the fingers of a user without the use of mechanical linkages. The user's movements are not restricted since the device does not have mechanical linkages, and the user can enjoy haptic interactions with virtual environments. In order to downsize the device, we designed our device on the basis of a hypothesis that kinesthetic sensations on the user's fingers alone are sufficient to represent the sensations of touch. We implemented a prototype device and performed an experiment to confirm the representational ability of our device. We also developed a prototype haptic augmented reality system, using which the user can see and touch a computer graphics object.

In order to realize haptic interaction (e.g., holding, pushing, and contacting the object) in virtual environment and mediated haptic communication with human beings (e.g., handshaking), the force feedback is required. Recently there has been a substantial need and interest in haptic displays, which can provide realistic and high fidelity physical interaction in virtual environment. The aim of our research is to implement a wearable haptic display for presentation of realistic feedback (kinesthetic stimulus) to the human arm. We developed a wearable device FlexTorque that induces forces to the human arm and does not require holding any additional haptic interfaces in the human hand. It is completely new technology for Virtual Reality that allows user to explore surroundings freely. The concept of Karate (empty hand) Haptics proposed by us is opposite to conventional interfaces (e.g., Wii Remote, SensAble's PHANTOM, SPIDAR [Murayama et al. 2004]) that require holding haptic interface in the hand, restricting thus the motion of the fingers in midair.

It is expected that the use of haptic sensors to measure the magnitude, direction, and distribution of a force will enable a robotic hand to perform dexterous manipulations. Therefore, we have developed a new type of finger-shaped haptic sensor that can measure a three-dimensional force vector field over a contact surface. The sensor consists of a transparent elastic body, two layers of internal blue and red markers, and a CCD camera to capture the movements of the markers. Using the elastic theory, we can calculate the force vector field from the captured movements of the markers. However, the elastic theory cannot be applied to the finger-shaped sensor because of its complicated shape. Therefore, we use actual measurements for the calibration in order to develop a prototype of the sensor. Then, we evaluate its basic performance. The result shows that the sensor performance can be improved further, and the sensor can be successfully used in a robotic hand.

For safety and operability of drivers while operating a vehicle, it is very important to obtain a wide field of vision. However, the space available for setting up windows is limited. Therefore, we propose a ";transparent cockpit,"; in which the image of a blind spot is displayed on the inner wall of the vehicle using a retro-reflective projection technology. In this system, the internal components of the vehicle, such as the doors and floor, are virtually transparent, and the blind spot is clearly visible, as observed from a window. In this paper, we describe the implementation of a prototype of the proposed system and demonstrate its effectiveness by an evaluation experiment.

The goal of this study is to develop a haptic interface for dexterous manipulation. To achieve this, we proposed electrotactile-kinesthetic integration. Our electrotactile display presents natural touch sensations of objects. In addition, this display is so small that it is considered to not affect the moving range of fingers. The haptic interface for dexterous manipulation is realized by mounting it on a kinesthetic display having a wide workspace. The electrotactile display on the integrated haptic interface is used actively. Therefore, we actively evaluated the performances of the electrotactile display on the single-fingered prototype system. The results revealed the present performance of the system. Subsequently, it also shows the possibility of further improving of the haptic interface by devising an electrotactile rendering method.

The paper focuses on design and joint impedance control of a new teleoperated robot arm enabling torque measurement in each joint by means of incorporation of devised optical torque sensors. When the contact of arm with an object occurs, joint impedance algorithm provides active compliance of corresponding robot arm joint. Thus, the whole structure of the manipulator can safely interact with unstructured environment. In the paper, we describe detailed design procedure of the 4-DOF robot arm and optical torque sensors. To extract force signal from measured data, the gravity compensation algorithm was elaborated and verified. The experimental results of joint impedance control show that proposed strategy provides safe interaction of entire structure of robot arm with human beings and ensures the collision avoidance.

It is necessary to naturally avoid singular configurations of each slave robot in order to achieve an ideal telexistence master-slave system. We propose a method for the avoidance of singular configurations by using intervening impedance. In this method, the parameters of the intervening impedance are adjusted by the manipulability ellipsoid. This method is applied to a master-slave robot. As a result, singular configurations are avoided by the proposed method.

This paper introduces a pervasive sensor system for nursing homes, where daily activities of elderly persons are monitored by pervasive sensors all the time. Deterioration in the quality of nursing care for the elderly has become one of the biggest problems in the aging society and the authors have been challenging the problem by sensors embedded in a nursing room. The sensors accumulate position information of a subject person and his wheelchair, then it is utilized for prompt assists for the subject and also used for obtaining his life log. In our experiments, we obtained position data of an elderly person for a month and a half and analyzed his sleeping hours and the number of times of going to a restroom. This paper presents the concept of the system, overview of the current system and experimental results obtained

We have developed a new type of master hand to control a dexterous slave robot hand for telexistence. Our master hand has two features. One is a compact exoskeleton mechanism called “circuitous joint,” which covers wide workspace of an operator’s finger. Another is the encounter-type force feedback that enables unconstrained motion of the operator’s finger and natural contact sensation. In this paper, the mechanism and control method of our master hand are introduced and experimental master-slave finger control is conducted.

We previously proposed a new 3D form display actuated by shape memory alloy (SMA), which is capable of displaying large scale objects sequentially. Based on our devised method, our display uses a 16×16 array of pin-rods with 30mm stroke at 5mm intervals. In this paper, we discuss the control method of the shape memory alloy with a simple matrix drive circuit. This type of drive circuit can dramatically reduce the number of switch. In addition, we propose a novel approach using linear programming, where the simulated time to display a whole shape is reduced by about 50% or more with a large number of actuators.

Artificial neural networks have been traditionally employed to learn and compute the inverse kinematics of a robotic arm. However, the inverse kinematics model of a typical robotic arm with joint limits is a multi-valued and discontinuous function. Because it is difficult for a multilayer neural network to approximate this type of function, an accurate inverse kinematics model cannot be obtained by using a single neural network. In order to overcome the difficulties of inverse kinematics learning, we propose a novel modular neural network system that consists of a number of expert modules, where each expert approximates a continuous part of the inverse kinematics function. The proposed system selects one appropriate expert whose output minimizes the expected position/orientation error of the end-effector of the arm. The system can learn a precise inverse kinematics model of a robotic arm with equal or more degrees of freedom than that of its end-effector. However, there are robotic arms with fewer degrees of freedom, where the system cannot learn their precise inverse kinematics model. We have adopted a modified Gauss-Newton method for finding the least-squares solution to address this issue. Through the modifications presented in this paper, the improved modular neural network system can obtain a precise inverse kinematics model of a general robotic arm.

We developed a robotic arm for a master-slave system to support "mutual telexistence," which realizes remote dexterous manipulation tasks and close physical communication with other people using gestures. In this paper, we describe the specifications of the experimental setup of the master-slave arm to demonstrate the feasibility of the mutual telexistence concept. We developed the master arm of a telexistence robot for interpersonal communication. The last degree of the 7-degree-of-freedom slave arm is resolved by placing a small orientation sensor on the operators arm. This master arm is made light and impedance control is applied in order to grant the operator as much freedom of movement as possible. For this development stage, we compared three control methods and confirmed that the impedance control method is the most appropriate to this system.

One of the key techniques for vision-based communication is omnidirectional stereo (omnistereo) imaging, in which stereoscopic images for an arbitrary horizontal direction are captured and presented according to the viewing direction of the observer. Although omnistereo models have been surveyed in several studies, few omnistereo sensors have actually been implemented. In this paper, a practical method for capturing omnistereo video sequences using rotating optics is proposed and evaluated. The rotating optics system consists of prism sheets, circular or linear polarizing films, and a hyperboloidal mirror. This system has two different modes of operation with regard to the separation of images for the left and right eyes. In the high-speed shutter mode, images are separated using postimage processing, while, in the low-speed shutter mode, the image separation is completed by optics. By capturing actual images, we confirmed the effectiveness of the methods.

Humans can perceive not only the magnitude but also the direction of force applied on the fingertip. When we grasp an object, the weight of it is felt through force that is parallel to the skin of the fingertip, which is how the object can be grasped without slipping. Focusing on this point, we have developed a tactile sensor that can measure a distribution of force vectors. The measurement method is as follows. The tactile sensor consists of a transparent elastic body, blue and red markers inside the elastic body, and a color CCD camera. An applied force on the elastic body results in movements of the markers, which are acquired by the CCD camera. The distribution of force vectors is calculated using this information. This paper reports experimental evaluation results concerning accuracy of determining position of markers, determination of magnitude and direction of force, spatial resolution, and calculation timing.

This paper describes a kind of active camouflage system named optical camouflage. Optical camouflage uses the retro-reflective projection technology, a projection-based augmented-reality system composed of a projector with a small iris and a retro-reflective screen. The object that needs to be made transparent is painted or covered with retro-reflective material. Then a projector projects the background image on it making the masking object virtually transparent.

An essential factor in understanding the motor learning capability of humans, is the coordinate transformation learning of the visual feedback controller. Although a number of learning models for the visual feedback controller have been proposed, none has been able to establish a definitive approach. In our previous work, we have suggested a learning model that uses disturbance noise and the feedback error signal to learn the human visual feedback controller's coordinate transformation. However, the model does not fully consider the time delay in the visual feedback control loop. This paper presents a modified learning model taking into account the time delay and the convergence properties of the model. Numerical simulations are presented to illustrate the effectiveness of the proposed approach.

Previous research on augmented reality has been mainly focused on augmentation of visual or acoustic information. However, humans can receive information not only through vision and acoustics, but also through haptics. Haptic sensation is very intuitive, and some researchers are focusing on making use of haptics in augmented reality systems. While most previous research on haptics is based on static data, such as that generated from CAD, CT, and so on, these systems have difficulty responding to a changing real environment in real time. In this paper, we propose a new concept for the augmented reality of haptics, the SmartTool. The SmartTool responds to the real environment by using real time sensor(s) and a haptic display. The sensor(s) on the SmartTool measure the real environment then send us that information through haptic sensation. Furthermore, we will describe the prototype system we have developed.

Inverse kinematics computation using an artificial neural network that learns the inverse kinematics of a robot arm has been employed by many researchers. However, the inverse kinematics system of typical robot arms with joint limits is a multivalued and discontinuous function. Since it is difficult for a well-known multilayer neural network to approximate such a function, a correct inverse kinematics model cannot be obtained by using a single neural network. In order to overcome the discontinuity of the inverse kinematics function, we proposed a novel modular neural network system that consists of a number of expert neural networks. Each expert approximates the continuous part of the inverse kinematics function. The proposed system uses the forward kinematics model for selection of experts. When the number of the experts increases, the computation time for calculating the inverse kinematics solution also increases without using the parallel computing system. In order to reduce the computation time, we propose a novel expert selection by using the performance prediction networks which directly calculate the performances of the experts.

We have proposed and developed a tactile display that uses electrical current as a stimulus. The main characterstic of this display is that each type of mechanoreceptor is stimulated separately so that "tactile primary colors" are defined. In this paper, we describe how we fabricated a 4/spl times/4 array electrode system to present surface roughness, patterns and relative motions.

We present an autostereoscopic display named TWISTER I (Telexistence Wide-view-angle Immersive STEReoscope, Model I), which is designed for a face-to-face telecommunication system called "mutual telexistence". By rotating display units that consist of LED arrays and a barrier around a viewer, TWISTER I can display panoramic stereoscopic images that can be observed without the use of special eyewear. This "glassless" feature is essential for applying this apparatus to mutual telexistence because eye contact is important in non-verbal communication.

The authors first analyze dynamic properties of rapidly adapting mechanoreceptors in order to derive a principle of tactile displays. It is shown that Meissner corpuscles with coiled axons and Pacinian corpuscles with layered lamellae are suited to detect equivoluminal distortion of skin. Combining these analyses with a model of a contact and a relative motion between an object and the skin, it is shown that a prerequisite for tactile displays is to generate sources of shear stress that can be spatially dispersed at the skin surface and temporally modulated with a stick-slip frequency determined by parameters of the object to be displayed. As a device that satisfies this prerequisite, we propose a tactile display using Surface Acoustic Waves (SAW). The roughness of the surface can be changed continuously by controlling the burst frequency of the SAW.

Inverse kinematics computation using an artificial neural network that learns the inverse kinematics of a robot arm has been employed by many researchers, However, conventional learning methodologies do not pay enough attention the the discontinuity of the inverse kinematics system of typical robot arms with joint limits. The inverse kinematics system of the robot arms is a multi-valued and discontinuous function. Since it is difficult for a well-known multi-layer neural network to approximate such a function, a correct inverse kinematics model for the end-effector's overall position and orientation cannot be obtained by using a single neural network. In order to overcome the discontinuity of the inverse kinematics function, we propose a modular neural network system for the inverse kinematics model learning. We also propose the online learning and control method for trajectory tracking.

In order to grasp an object, we need to solve the inverse kinematics problem, i.e., the coordinate transformation from the visual coordinates to the joint angle vector coordinates of the arm. In human motion control, the learning of the hand position error feedback controller in the inverse kinematics solver is important. Although several models of coordinate transformation learning have been proposed, they suffer from a number of drawbacks. This paper proposes a novel model of the coordinate transformation learning of the human visual feedback controller by using disturbance noise and feedback error signal. The feasibility of the proposed model is illustrated using numerical simulations.

In order to learn an inverse system of a plant, the learning method which inputs the output of the plant to the learner and uses the input of the plant as the desired output signal for the learner has been used in many researches. This learning method for learning control by using a neural network is called "direct inverse modeling" (DIM). Jordan (1995) regarded DIM as a purely off-line learning method and pointed out that DIM is not goal-directed. However, some researches have proposed a different way of using DIM. We deal with online DIM, which is the simultaneous or alternate execution of the plant control and the inverse model learning by DIM. The learning properties of online DIM is completely different from that of off-line DIM. This paper shows that online DIM becomes goal-directed under certain conditions.

The purpose of this research is to demonstrate the influence that a water dome has on a person. The following approach was used for this study: (1) a water display was used as a projection screen; (2) the illusion of being under water was created; and (3) experiments with water domes of various sizes were carried out. The water display is a head mounted type water membrane. When the display is projected, the internal surface acts as a concave mirror that can reflect the hand and face of a participant. The water dome is a hemisphere 8 m in diameter. Participants were allowed to see the real world through a transparent wall of water on which computer-generated images were projected. A water dome has the advantage of engaging a participant's five senses as it enhances the surrounding environment. We developed a water dome that incorporates the points of view of both the artist and the engineer.

Proposes a novel visuo-haptic display using a head-mounted projector (HMP) with X'tal Vision (Crystal Vision) optics. Our goal is to develop a device which enables an observer to touch a virtual object just as it is seen. We describe in detail the design of an HMP with X'tal Vision, which is very suitable for augmented reality. For instance, the HMP makes the occlusion relationship between the virtual and the real environments nearly correct. Accordingly, the user can observe his/her real hand with the virtual objects. Furthermore, the HMP reduces eye fatigue because of the low inconsistency of accommodation and convergence. Therefore, we applied HMP-model 2 to a visuo-haptic display using a camouflage technique. This technique, called optical camouflage, makes an obstacle object, such as a haptic display, become translucent. With this method, a user can observe a stereoscopic virtual object with a nearly correct occlusion relationship between the virtual and the real environments, and he can actually feel the object.

Projection-based visual display systems are expected to be effective platforms for virtual reality (VR) applications in which the displayed images are generated by computer graphics using 3D models of virtual worlds. However, these kinds of visual displays, as well as other kinds of fixed screen-based displays, such as various head-tracked displays (HTDs) and conventional CRT displays, have not been utilized to achieve precise telexistence in a real environment, which requires appropriate stereoscopic video images corresponding to the operator's head motion. We found that the time-varying off-axis projection required in these systems has prevented fixed screen-based displays from being used for telexistence, as ordinary cameras only have fixed and symmetric fields of view about the optical axis. After evaluating the problem, a method to realize a live video-based telexistence system with a fixed screen is proposed, aiming to provide the operator with a natural 3D sensation of presence. The key component of our method is a feature that keeps the orientation of the cameras fixed, regardless of the operator's head motion. Such a feature was implemented by designing a constant-orientation link mechanism.

A computational approach is applied to the problem of controlling fixation of haptic perception during general motion. Estimation of hardness and curvatures of the environment from a single pressure image is shown to be ill-posed, and an active observation is demonstrated making the estimation well-posed. Also proposed is an algorithm for haptic servoing for the active finger-pad to fix its high-resolution region on the surface of contact.

Inverse kinematics computation using an artificial neural network that learns the inverse kinematics of a robot arm has been employed by many researchers. However, conventional learning methodologies do not pay enough attention to the discontinuity of the inverse kinematics system of typical robot arms with joint limits. The inverse kinematics system of the robot arms, including a human arm with a wrist joint, is a multivalued and discontinuous function. Since it is difficult for a well-known multilayer neural network to approximate such a function, a correct inverse kinematics model for the end-effector's overall position and orientation cannot be obtained by the conventional methods. In order to overcome the drawbacks of the inverse kinematics solver consisting of a single neural network, we propose a novel modular neural network architecture for the inverse kinematics model learning.

Most algorithms currently proposed for sensor integration implicitly assume that the signals from all sensors are synchronized, but this does not hold in real situations. The paper proposes to handle time-variant sensor combination systems by a sequence, and formulates a method to verify that the designed sequence is stable, together with a method to predict theoretically the performance of the designed sequence based on the mutual information rate. This enables a sequence of sensor combinations to be designed rationally.

A method for controlling a manipulator with passive joints, which have no actuators, in operational space is presented. The equation of motion is described in terms of operational coordinates. The coordinates are separated into active and passive components. The acceleration of the active components can be arbitrarily adjusted by using the coupling characteristics of manipulator dynamics. This method is also extended to path tracking control of a manipulator with passive joints. A desired path is geometrically specified in operational space. The position of the manipulator is controlled to follow the path. In this method, a path coordinate system based on the path is defined in operational space. The path coordinates consist of a component parallel to the path and components normal to the path. The acceleration of the components normal to the path is controlled according to feedback based on tracking error by using the dynamic coupling among the components. This in turn keeps the manipulator on the path. The effectiveness of the method is verified by experiments using a two-degree-of-freedom manipulator with a passive joint.>

When the minimum-time trajectory of a manipulator along a geometrically prescribed path is planned taking into consideration the manipulator's dynamics and actuator's torque limits, at least one of the joints should be at the torque limit. The execution of such a trajectory by a conventional feedback control scheme results in torque saturation. Consequently, the tracking error cannot be suppressed and the manipulator may deviate from the desired path. In this paper, the author's propose a feedback control method for path tracking which takes the torque saturation into account. Based on the desired path, a coordinate system called path coordinates is defined. The path coordinates are composed of the component along the path and the components normal to the path. The equation of motion is described in terms of the path coordinates. Control of the components normal to the path is given priority in order to keep the motion of the manipulator on the path. Simulations of a two-degree-of-freedom manipulator show the effectiveness of this method. >

A method for controlling the tip position of a manipulator with passive joints, which have no actuators, in an operational coordinate space is presented. The equations of motion are described in terms of operational coordinates. The coordinates are separated into controlled coordinates and compensating coordinates. The accelerations of the controlled coordinates can be arbitrarily adjusted by using dynamic coupling of the manipulator. The effectiveness of the method is verified by experiments using a two-degree-of-freedom manipulator with a passive joint. The experiments show that the controlled coordinates of the position of the manipulator can be controlled precisely by means of the proposed method.>

A method is proposed of path tracking control of a manipulator with passive joints, i.e. having no actuators. A desired path is geometrically specified in operational space. The position of the manipulator is controlled to follow the desired path. In this method, a path coordinate system based on the desired path is defined in operational space. The path coordinates consist of a component parallel to the desired path and components normal to the desired path. The equation of motion of the manipulator is described in terms of the path coordinates. The acceleration of the components normal to the desired path is controlled according to the feedback based on tracking errors by using the dynamic coupling among the components. This in turn keeps the manipulator on the desired path. Results of path tracking experiments using a two-degree-of-freedom manipulator with a passive joint are presented.>

A method of controlling the position of a manipulator composed of active and passive joints is described. The active joints have actuators and position sensors. The passive joints have holding brakes instead of actuators. While the brakes are released, the passive joints are indirectly controlled by the motion of the active joints using the coupling characteristics of manipulator dynamics. While the brakes are engaged, the passive joints are fixed and the active joints are controlled. The position of the manipulator is controlled by combining these two control modes. The basic principle of the control method and the conditions that ensure the controllability of the passive joints are described. An algorithm for point-to-point control of the manipulator is presented. The feasibility of the method is demonstrated by simulations for a manipulator with two degrees of freedom.>

Because of the size of walking machines, scaled-down models having geometrical similarity can be used instead of full-scale models to measure energetic efficiency. First, to cope with this situation, five nondimensional parameters that control energetic efficiency Of walking machines are introduced for nine physical parameters by applying dimensional analysis techniques. Next, in order to check those influences on energetic efficiency, computer simulations are performed for an n-legged walking model. In these simulations, the influence on ground reaction forces is considered to satisfy the approximate balance of forces and moments when an overall walking machine assembly is assumed as a free body. Resultantly, several new facts are acquired from the viewpoint of similarity law.