We propose a novel interface called Twinkle for interacting with an arbitrary physical surface using a handheld projector and a camera. When a user flashes a projection light on an object, the projected images react as if the user touched the object with the light. The handheld device recognizes the features of the physical environment and displays images and sounds that are generated in real-time according to the user's motion and collisions of projected images with objects. We realize this system by using several image-processing techniques and a collision detection algorithm. We also use an acceleration sensor to compensate for the image processing. In this paper, we explain the principle of interacting with a physical surface. Then, we describe the implementation of the prototype system and some application examples.

We propose a novel grip-type master hand called MeisterGRIP that measures grip force in terms of a force vector distribution. This device is expected to allow intuitive robot manipulation using vision-based haptic-sensing technology. Furthermore, it can be used for general-purpose manipulation and is tolerant to individual differences in hand size and grasping posture. We constructed MeisterGRIP and evaluated the accuracy of the measured grip force. Furthermore, we constructed and exhibited a complete robot manipulation system using MeisterGRIP to demonstrate the possibility of using MeisterGRIP as a general-purpose master hand.

We proposed a pen-shaped handheld haptic display that provides kinesthetic sensations to the fingers of a user without the use of mechanical linkages. The user's movements are not restricted since the device does not have mechanical linkages, and the user can enjoy haptic interactions with virtual environments. In order to downsize the device, we designed our device on the basis of a hypothesis that kinesthetic sensations on the user's fingers alone are sufficient to represent the sensations of touch. We implemented a prototype device and performed an experiment to confirm the representational ability of our device. We also developed a prototype haptic augmented reality system, using which the user can see and touch a computer graphics object.

In order to realize haptic interaction (e.g., holding, pushing, and contacting the object) in virtual environment and mediated haptic communication with human beings (e.g., handshaking), the force feedback is required. Recently there has been a substantial need and interest in haptic displays, which can provide realistic and high fidelity physical interaction in virtual environment. The aim of our research is to implement a wearable haptic display for presentation of realistic feedback (kinesthetic stimulus) to the human arm. We developed a wearable device FlexTorque that induces forces to the human arm and does not require holding any additional haptic interfaces in the human hand. It is completely new technology for Virtual Reality that allows user to explore surroundings freely. The concept of Karate (empty hand) Haptics proposed by us is opposite to conventional interfaces (e.g., Wii Remote, SensAble's PHANTOM, SPIDAR [Murayama et al. 2004]) that require holding haptic interface in the hand, restricting thus the motion of the fingers in midair.

It is expected that the use of haptic sensors to measure the magnitude, direction, and distribution of a force will enable a robotic hand to perform dexterous manipulations. Therefore, we have developed a new type of finger-shaped haptic sensor that can measure a three-dimensional force vector field over a contact surface. The sensor consists of a transparent elastic body, two layers of internal blue and red markers, and a CCD camera to capture the movements of the markers. Using the elastic theory, we can calculate the force vector field from the captured movements of the markers. However, the elastic theory cannot be applied to the finger-shaped sensor because of its complicated shape. Therefore, we use actual measurements for the calibration in order to develop a prototype of the sensor. Then, we evaluate its basic performance. The result shows that the sensor performance can be improved further, and the sensor can be successfully used in a robotic hand.

For safety and operability of drivers while operating a vehicle, it is very important to obtain a wide field of vision. However, the space available for setting up windows is limited. Therefore, we propose a ";transparent cockpit,"; in which the image of a blind spot is displayed on the inner wall of the vehicle using a retro-reflective projection technology. In this system, the internal components of the vehicle, such as the doors and floor, are virtually transparent, and the blind spot is clearly visible, as observed from a window. In this paper, we describe the implementation of a prototype of the proposed system and demonstrate its effectiveness by an evaluation experiment.

The goal of this study is to develop a haptic interface for dexterous manipulation. To achieve this, we proposed electrotactile-kinesthetic integration. Our electrotactile display presents natural touch sensations of objects. In addition, this display is so small that it is considered to not affect the moving range of fingers. The haptic interface for dexterous manipulation is realized by mounting it on a kinesthetic display having a wide workspace. The electrotactile display on the integrated haptic interface is used actively. Therefore, we actively evaluated the performances of the electrotactile display on the single-fingered prototype system. The results revealed the present performance of the system. Subsequently, it also shows the possibility of further improving of the haptic interface by devising an electrotactile rendering method.

The paper focuses on design and joint impedance control of a new teleoperated robot arm enabling torque measurement in each joint by means of incorporation of devised optical torque sensors. When the contact of arm with an object occurs, joint impedance algorithm provides active compliance of corresponding robot arm joint. Thus, the whole structure of the manipulator can safely interact with unstructured environment. In the paper, we describe detailed design procedure of the 4-DOF robot arm and optical torque sensors. To extract force signal from measured data, the gravity compensation algorithm was elaborated and verified. The experimental results of joint impedance control show that proposed strategy provides safe interaction of entire structure of robot arm with human beings and ensures the collision avoidance.

It is necessary to naturally avoid singular configurations of each slave robot in order to achieve an ideal telexistence master-slave system. We propose a method for the avoidance of singular configurations by using intervening impedance. In this method, the parameters of the intervening impedance are adjusted by the manipulability ellipsoid. This method is applied to a master-slave robot. As a result, singular configurations are avoided by the proposed method.

We have developed a new type of master hand to control a dexterous slave robot hand for telexistence. Our master hand has two features. One is a compact exoskeleton mechanism called “circuitous joint,” which covers wide workspace of an operator’s finger. Another is the encounter-type force feedback that enables unconstrained motion of the operator’s finger and natural contact sensation. In this paper, the mechanism and control method of our master hand are introduced and experimental master-slave finger control is conducted.

We previously proposed a new 3D form display actuated by shape memory alloy (SMA), which is capable of displaying large scale objects sequentially. Based on our devised method, our display uses a 16×16 array of pin-rods with 30mm stroke at 5mm intervals. In this paper, we discuss the control method of the shape memory alloy with a simple matrix drive circuit. This type of drive circuit can dramatically reduce the number of switch. In addition, we propose a novel approach using linear programming, where the simulated time to display a whole shape is reduced by about 50% or more with a large number of actuators.

We developed a robotic arm for a master-slave system to support "mutual telexistence," which realizes remote dexterous manipulation tasks and close physical communication with other people using gestures. In this paper, we describe the specifications of the experimental setup of the master-slave arm to demonstrate the feasibility of the mutual telexistence concept. We developed the master arm of a telexistence robot for interpersonal communication. The last degree of the 7-degree-of-freedom slave arm is resolved by placing a small orientation sensor on the operators arm. This master arm is made light and impedance control is applied in order to grant the operator as much freedom of movement as possible. For this development stage, we compared three control methods and confirmed that the impedance control method is the most appropriate to this system.

Humans can perceive not only the magnitude but also the direction of force applied on the fingertip. When we grasp an object, the weight of it is felt through force that is parallel to the skin of the fingertip, which is how the object can be grasped without slipping. Focusing on this point, we have developed a tactile sensor that can measure a distribution of force vectors. The measurement method is as follows. The tactile sensor consists of a transparent elastic body, blue and red markers inside the elastic body, and a color CCD camera. An applied force on the elastic body results in movements of the markers, which are acquired by the CCD camera. The distribution of force vectors is calculated using this information. This paper reports experimental evaluation results concerning accuracy of determining position of markers, determination of magnitude and direction of force, spatial resolution, and calculation timing.

This paper describes a kind of active camouflage system named optical camouflage. Optical camouflage uses the retro-reflective projection technology, a projection-based augmented-reality system composed of a projector with a small iris and a retro-reflective screen. The object that needs to be made transparent is painted or covered with retro-reflective material. Then a projector projects the background image on it making the masking object virtually transparent.

We have proposed and developed a tactile display that uses electrical current as a stimulus. The main characterstic of this display is that each type of mechanoreceptor is stimulated separately so that "tactile primary colors" are defined. In this paper, we describe how we fabricated a 4/spl times/4 array electrode system to present surface roughness, patterns and relative motions.

Proposes a novel visuo-haptic display using a head-mounted projector (HMP) with X'tal Vision (Crystal Vision) optics. Our goal is to develop a device which enables an observer to touch a virtual object just as it is seen. We describe in detail the design of an HMP with X'tal Vision, which is very suitable for augmented reality. For instance, the HMP makes the occlusion relationship between the virtual and the real environments nearly correct. Accordingly, the user can observe his/her real hand with the virtual objects. Furthermore, the HMP reduces eye fatigue because of the low inconsistency of accommodation and convergence. Therefore, we applied HMP-model 2 to a visuo-haptic display using a camouflage technique. This technique, called optical camouflage, makes an obstacle object, such as a haptic display, become translucent. With this method, a user can observe a stereoscopic virtual object with a nearly correct occlusion relationship between the virtual and the real environments, and he can actually feel the object.