Analyzing time-series changes of the contact surface by touching is important to elucidate the touching skills of Humanitude as one of the pervasive multimodal comprehensive care methodologies. For the analysis, there is a frustrated total internal reflection (FTIR) method to capture contact surfaces on a transparent flat plate by a camera. However, this conventional flat plate is far from the actual surfaces of care receivers because the surfaces of humans consist of curved shapes. In this paper, we propose an FTIR sensing system using a transparent curved plate to capture more ideal contact states with a surface shape more similar to the human body. We collect the contact surface data of the beginning of touching motions by Humanitude experts and novices using the FTIR sensing system with the curved and flat plates. Then, we compare the data by the experts and novices in terms of time-series contact areas and the quantitative indices and subjective evaluation and discuss the analysis results. Through these experiments, we confirm that the proposed system has the potential for the novices to perform more correctly the beginning of Humanitude's touching motions.

For assembly tasks, it is essential to fix target parts firmly and accurately estimate their poses. Several rigid jigs for individual parts are frequently used in assembly factories to achieve a precise and time-efficient product assembly. However, providing customized jigs is time-consuming. In this study, to address the lack of versatility in the shapes for which jigs can be used, we developed a flexible jig with a soft membrane including transparent beads and oil with a tuned refractive index. The bead-based jamming transition was accomplished by discharging only the oil, enabling the part to be firmly fixed. Because the two cameras under the jig can capture membrane shape changes, we proposed a sensing method to estimate the orientation of the part based on the behaviors of markers created on the jig's inner surface. Through estimation experiments, the proposed system can estimate the orientation of a cylindrical object with a diameter larger than 50 mm and an RMSE of less than 3°.

To design a general-purpose assembly robot system that can handle objects of various shapes, we propose a soft jig that fits to the shapes of assembly parts. The functionality of the soft jig is based on a jamming gripper developed in the field of soft robotics. The soft jig has a bag covered with a malleable silicone membrane, which has high friction, elongation, and contraction rates for keeping parts fixed. The bag is filled with glass beads to achieve a jamming transition. We propose a method to configure parts-fixing on the soft jig based on contact relations, reachable directions, and the center of gravity of the parts that are fixed on the jig. The usability of the soft jig was evaluated in terms of the fixing performance and versatility for various shapes and postures of parts.

To generate assembly sequences that robots can easily handle, this study tackled assembly sequence generation (ASG) by considering two tradeoff objectives: (1) insertion conditions and (2) degrees of the constraints affecting the assembled parts. We propose a multi-objective genetic algorithm to balance these two objectives. Furthermore, we extend our previously proposed 3D computer-aided design (CAD)-based method for extracting three types of two-part relationship matrices from 3D models that include deformable parts. The interference between deformable and other parts can be determined using scaled part shapes. Our proposed ASG can produce Pareto-optimal sequences for multi-component models with deformable parts such as rubber bands, rubber belts, and roller chains. We further discuss the limitation and applicability of the generated sequences to robotic assembly.

A robust, easy-to-deploy robot for service tasks in a real environment is difficult to construct. Record-and-playback (R&P) is a method used to teach motor-skills to robots for performing service tasks. However, R&P methods do not scale to challenging tasks where even slight changes in the environment, such as localization errors, would either require trajectory modification or a new demonstration. In this paper, we propose a Sequence-to-Sequence (Seq2Seq) based neural network model to generate robot trajectories in configuration space given a context variable based on real-world measurements in Cartesian space. We use the offset between a target pose and the actual pose after localization as the context variable. The model is trained using a few expert demonstrations collected using teleoperation. We apply our proposed method to the task of toilet cleaning where the robot has to clean the surface of a toilet bowl using a compliant end-effector in a constrained toilet setting. In the experiments, the model is given a novel offset context and it generates a modified robot trajectory for that context. We demonstrate that our proposed model is able to generate trajectories for unseen setups and the executed trajectory results in cleaning of the toilet bowl.

For a robotic gripper to perform as well as the human hand, the performance of the tactile sensing and grasping of the gripper must be improved. In this study, a parallel gripper with a universal fingertip device that combines both object holding using jamming transition and optical sensing is proposed. By using a transparent material along with the jamming transition, both a tactile sense is achieved via optical sensing and the holding stability is improved by tightly restraining the object. Because the hardness of the contact region of the membrane can be adjusted, the gripper can grasp heavy objects as well as light objects. In this paper, the structure of the developed gripper and its grasping characteristics are described, and the experimental results from grasping and sensing are presented. It is experimentally shown that the proposed gripper is capable of both grasping fragile objects (such as an egg and tofu) by using tactile sensing and grasping dumbbells (with weights between 1-3 kg) by using jamming transition. Additionally, grasping a plastic cup, the proposed gripper demonstrated the potential of combining tactile sensing and more-constrained grasping.

The appearance of android robots is very similar to that of human beings. From their appearance, we expect that androids might provide us with high-level communication. The imitation of human behavior gives us the feeling of natural behavior even if we do not know what drives high-level communication. In this paper, we evaluate the imitation of human eye behavior by an android. We consider that the android imitates human eye behavior while explaining some research topic and a person acts as a listener. Then, we construct a method to imitate the eye behavior obtained from eye trackers. For the evaluation, we asked seventeen male subjects for their subjective evaluation and compared the imitation with an android that controlled eye-contact duration and eyeblinks by editing the imitation or programming rule-based behavior. From the results, we found out that 1) the rule-based behaviors kept human-likeness, 2) 3-second eye contact obtained better scores regardless of the imitation-based or rule-based eye behavior, and 3) the subjects might regard the longer eyeblinks as voluntary eyeblinks, with the intention to break eye contacts.

As sensing and image recognition technologies advance, the environments where service robots operate expand into human-centered environments. Since the roles of service robots depend on the user situations, it is important for the robots to understand human intentions. Gaze information, such as gazed objects (i. e., the objects humans are looking at) can help to understand the users' intentions. In this paper, we propose a real-time gazed object identification method from RGBD images captured by a camera mounted on a mobile service robot. First, we search for the candidate gazed objects using state-of-the-art, real-time object detection. Second, we estimate the human face direction using facial landmarks extracted by a real-time face detection tool. Then, by searching for an object along the estimated face direction, we identify the gazed object. If the gazed object identification fails even though a user is looking at an object, i. e., has a fixed gaze direction, the robot can determine whether the object is inside or outside the robot's view based on the face direction, and, then, change its point of view to improve the identification. Finally, through multiple evaluation experiments with the mobile service robot Pepper, we verified the effectiveness of the proposed identification and the improvement of the identification accuracy by changing the robot's point of view.

In this work we propose a novel approach to remove undesired objects from RGB-D sequences captured with freely moving cameras, which enables static 3D reconstruction. Our method jointly uses existing information from multiple frames as well as generates new one via inpainting techniques. We use balanced rules to select source frames; local homography based image warping method for alignment and Markov random field (MRF) based approach for combining existing information. For the left holes, we employ exemplar based multi-view inpainting method to deal with the color image and coherently use it as guidance to complete the depth correspondence. Experiments show that our approach is qualified for removing the undesired objects and inpainting the holes.

We propose a new neural network called Temporal-enhanced Convolutional Network (T-CN) for video-based person reidentification. For each video sequence of a person, a spatial convolutional subnet is first applied to each frame for representing appearance information, and then a temporal convolutional subnet links small ranges of continuous frames to extract local motion information. Such spatial and temporal convolutions together construct our T-CN based representation. Finally, a recurrent network is utilized to further explore global dynamics, followed by temporal pooling to generate an overall feature vector for the whole sequence. In the training stage, a Siamese network architecture is adopted to jointly optimize all the components with losses covering both identification and verification. In the testing stage, our network generates an overall discriminative feature representation for each input video sequence (whose length may vary a lot) in a feed-forward way, and even a simple Euclidean distance based matching can generate good re-identification results. Experiments on the most widely used benchmark datasets demonstrate the superiority of our proposal, in comparison with the state-of-the-art.

The universal gripper has attracted attention due to its simple structure and advanced grasping ability for irregularly shaped objects. In this research, we propose a novel design for a granular-jamming-based gripper which uses a transparent filling and a semi-transparent membrane to allow optical sensing to detect both deformation of the membrane and the object being grasped. By adjusting the refractive index of an oil mixture to the refractive index of the granular bodies, we produced a fully transparent filling that allows the use of a camera inside the universal gripper. In this paper, we present the materials and development of our prototype, and describe the experimental confirmation of the prototype's performance. We showed that our prototype was able to grasp cylindrical and rectangular objects between 10 to 70 mm length while also tracking the deformation of the gripper.

Since the appearance of androids is very similar to the appearance of humans, androids are expected to perform human-like interactions. Among these interactions, telling a lie is especially interesting since such an interaction is not always malicious and is employed due to social politeness and to protect self-esteem of the person who tells a lie. In this paper, we develop and evaluate an android that performs humanlike subconscious behaviors when telling a lie in a simple game. For this purpose, we design a game to evaluate the effectiveness of such behaviors. Based on a literature review, we devise a strategy to implement such behaviors on the android. By observing the game being played between two subjects, we analyze and implement the behaviors obtained. The experimental results indicated that the android exhibiting the human-like subconscious behaviors gave users an impression of higher affinity, activity, and honesty. Finally, we concluded that playing games with the android was more amusing for subjects than simple interaction.

Robot usage in the fields of human support and healthcare is expanding. Robotic devices to assist humans in the self-feeding task have been developed to help patients with limited mobility in the upper limbs but the acceptance of these robots has been limited. In this work, we investigate how to quantitatively evaluate the comfort of an eating assistive device by estimating the interaction forces between the human and the robot when eating. Rather than using expensive or commercially unavailable devices to directly measure the forces involved in feeding, we use an accelerometer to estimate these forces, which are calculated using a previously observed estimation of the system mass and the measured acceleration during the feeding process. We experimentally verify our concept with a commercially-available eating assistive device and a human subject. The evaluation results demonstrate the feasibility of our approach.

Image tagging is a well known challenge in image processing. It is typically addressed through multi-instance multi-label (MIML) classification methodologies. Convolutional Neural Networks (CNNs) possess great potential to perform well on MIML tasks, since multi-level convolution and max pooling coincide with the multi-instance setting and the sharing of hidden representation may benefit multi-label modeling. However, CNNs usually require a large amount of carefully labeled data for training, which is hard to obtain in many real applications. In this paper, we propose a new approach for transferring pre-trained deep networks such as VGG16 on Imagenet to small MIML tasks. We extract features from each group of the network layers and apply multiple binary classifiers to them for multi-label prediction. Moreover, we adopt an L1-norm regularized Logistic Regression (L1LR) to find the most effective features for learning the multi-label classifiers. The experiment results on two most-widely used and relatively small benchmark MIML image datasets demonstrate that the proposed approach can substantially outperform the state-of-the-art algorithms, in terms of all popular performance metrics.

Many studies of hand motion recognition using a surface electromyogram (sEMG) have been conducted. However, it is difficult to get the activity of deep layer muscles from an sEMG. The pronation and supination of the forearm are caused by the activities of deep layer muscles. These motions are important in grasping and manipulating daily objects. We think it is possible to accurately recognize hand motions from the activity of the deep layer muscles using the forearm deformation. Forearm deformation is caused by a complex motion of the surface and deep layer muscles, tendons, and bones. In this study, we propose a novel hand motion recognition method based on measuring forearm deformation with a distance sensor array. The distance sensor array is designed based on a 3D model of the forearm. It can measure small deformations because the shape of the array is designed to fit the neutral position of the forearm. A Support Vector Machine (SVM) is used to recognize seven types of hand motion. Two types of features are extracted for the recognition based on the time difference of the forearm deformation. Using the proposed method, we perform hand motion recognition experiments. The experimental results showed that the proposed method correctly recognized hand motions caused by the activity of both surface and deep layer muscles, including the pronation and supination of the forearm. Moreover, the hand opening of small deformation motions was correctly recognized.

Moving to the location designated by the user is the most fundamental task for a mobile robot. Remote control is one of the effective solutions to navigate the robot to the target location, but the user suffers from the burden to continuously concentrate on the remote control. As the result, several users are required in accordance with the number of robots to operate. In this paper, we propose the remote control system that uses touch panel interface, simultaneous localization and mapping (SLAM), and motion planning to achieve autonomy of mobile robots. To navigate the robot in the proposed system, the user only designates the destination and via-points by touching on the map estimated by the SLAM. After receiving the user's input, the Rapidly-exploring Random Tree (RRT) generates the feasible path to the destination using the estimated map. The effectiveness of the proposed system is verified through experiments where multiple mobile robots are remotely controlled.

Unlike conventional portable eye-tracking methods that estimate the position of the mounted camera using 2-D image coordinates, the techniques that are proposed here present richer information about person's gaze when moving over a wide area. They also include visualizing scanpaths when the user with a head-mounted device makes natural head movements. We employ a Visual SLAM technique to estimate the head pose and extract environmental information. When the person's head moves, the proposed method obtains a 3-D point-of-regard. Furthermore, scanpaths can be appropriately overlaid on image sequences to support quantitative analysis. Additionally, a 3-D environment is employed to detect objects of focus and to visualize an attention map.

The Human-Robot Interaction gets increasingly closer. In consequence, human safety has become a key issue for the success of the symbiosis between humans and robots. When the minimum distance between a human and a robot is too short, it can be naturally considered that the probability of a collision increases. Therefore, we consider that the robot should increase the distance to the human when the human is getting closer. We propose Withdrawal strategy as a method that aims to increase the distance by moving the end-effector not only away from the human but also to a parking position that can be previously assessed to be safer. To withdraw the end-effector, we use a virtual force model consisting of two virtual forces: a repelling force exerted by the human and an attractive force exerted by the parking position. We carry out experiments using a human-sized humanoid robot and five human subjects, and report the task completion time to evaluate the efficiency of the robot when performing a simple task.

There are body-powered hooks and myoelectric prosthetic hands that trans-radial amputees can use for work. Though the body-powered hooks have good workability for complex operations, the design of the hook is unappealing and the harness is cumbersome. The myoelectric prosthetic hand has a natural appearance similar to the human hand and intuitive operability using a myoelectric control system. However, it is expensive and heavy. Because of these problems associated with prostheses for work, many amputees use cosmetic prostheses. In this paper, we report a lightweight, low-cost electric trans-radial prosthesis with three opposed fingers. A simple mechanism to control the fingers by a linear actuator contributes to its good workability, lightweight, and low-cost. An operation system using an inexpensive distance sensor allows intuitive operability equivalent to the myoelectric control system. A socket makes the prosthesis easily removable. The total weight of the hand and socket is 300 g, and both can be produced with a 3D printer. An evaluation using the Southampton Hand Assessment Procedure (SHAP) demonstrated that an amputee was able to operate abstract objects which require six types of grasps with the developed prosthesis.

In this paper, we propose a producing method of softness sensation using device vibration. The basic idea of the softness producing method is to provide a softness illusion of stimulating cutaneous mechanoreceptor by vibration. A prototype device is developed to verify effect of parameters of the vibration: frequency, amplitude and wave shape, on softness sensation. We create a database which indicates relationship between softness sensation and the vibration parameters to control the prototype device. Then subjective experiment is conducted for validation of the softness producing method. Experimental result shows that the proposing method can be used for softness sensation producing with valid resolution.

Natural body gesturing and speech dialogue, is crucial for human-robot interaction (HRI) and human-robot symbiosis. Real interaction is not only with one-to-one communication but also among multiple people. We have therefore developed a system that can adjust gestures and facial expressions based on a speaker's location or situation for multi-party communication. By extending our already developed real-time gesture planning method, we propose a gesture adjustment suitable for human demand through motion parameterization and gaze motion planning, which allows communication through eye-to-eye contact. We implemented the proposed motion planning method on an android Actroid-SIT and we proposed to use a Key-Value Store to connect the components of our systems. The Key-Value Store is a high-speed and lightweight dictionary database with parallelism and scalability. We conducted multi-party HRI experiments for 1,662 subjects in total. In our HRI system, over 60% of subjects started speaking to the Actroid, and the residence time of their communication also became longer. In addition, we confirmed our system gave humans a more sophisticated impression of the Actroid.

Natural body gesture, as well as speech dialog, is crucial for human-robot interaction and human-robot symbiosis. We have already proposed a real-time gesture planning method. In this paper, we afford this method more flexibility by adding motion parameterization function. Especially in multi-person HRI, this function becomes more important because of its adaptation to changes of a speaker's and/or object's locations. We implement our method for multi-person HRI system on the android Actroid-SIT, and conduct two experiments for estimating the precision of gestures and the human impressions about the Actroid. Through these experiments, we confirmed our method gives humans a more sophisticated impressions.

In this paper, we propose a method for semantic motion retrieval in large data sets of human motions to classify body gestures automatically. This method extracts spatio-temporal features from the motions by expressing them in frequency domain. And these features are transformed into the Bag-of-words representation to accelerate the calculation and to emphasize the semantic aspect. The method is inspired by techniques of natural language processing or image processing. We conducted experiments for evaluating the performance of the motion classification using data sets captured by a motion capture system. Through the experiments, we confirmed that our method improves the performance of the motion classification and reduces the computational time drastically.

Many hand motion recognition methods using Electromyogram (EMG) signals have been developed. Because most studies used only surface EMG signals from a forearm, available motion-related information was limited. In this paper, we report a SVM (Support Vector Machine) based hand motion recognition method using hybrid sensors. The hybrid sensor consists of an EMG sensor and an optical distance sensor, and can measure myoelectric activities and distance between the sensor and the skin surface at the same time. It is expected that distance changes caused by muscle elevation compensate the limited information derived from myoelectric activities. To examine the effectiveness of our method, we performed hand motion recognition experiments with four subjects. Experimental results showed that our method using hybrid sensors can improve motion recognition accuracy compared to when using only EMG signals.

The quantitative evaluation of product usability is important for product design. In this paper, we propose an evaluation system for product usability using a human mimetic robot hand with a tendon skeletal model. The proposed system enables us to evaluate product usability of a physical object, such as a prototype. To realize the system, we developed a robot hand which has a human mimetic structure. The design concepts of the robot hand are to imitate the human finger size, degree of freedom and fingertip structure. Ease of control is also characteristic of the robot hand. We propose to use tendon forces to evaluate the product usability. The joint torques during manipulation of the product are obtained from sensors on the robot hand. The tendon forces are calculated from the joint torques using the tendon skeletal model. Precise imitation of human finger structure makes the sensed joint torques similar to those of humans. The obtained joint torques are expected to be similar to those of humans. To evaluate the effectiveness of the proposed system, we conduct the experiments where the robot finger pushes cell phone button to evaluate its usability. The experimental result is verified by comparing with the existing questionnaire results from Ikeda [1]. The calculation score of the proposed system reflects the questionnaire result. These results indicate that the proposed system can quantitatively evaluate the actual product usability.

Androids are potentially required to show human-like behavior, because their appearance resembles humans' physical features. Therefore, we propose a gaze motion planning method. Within this method, we control the convergence of eyes and the ratio of eye angle to head angle, which leads to a more precise estimation of gaze direction. We implemented our method on the android Actroid-SIT and conducted experiments for evaluation of the effects of our method. Through these experiments, we achieved a common guidance for androids when planning more precise gaze motion.

In this paper, we propose a novel haptic device that gives weight and friction illusions. Sensing a slip condition at a fingertip plays an important role to estimate the weight and friction properties of an object. This fact suggests that controlling the contact condition could yield weight and friction illusions. Based on this idea, a prototype device was developed that controls the contact surface between a fingertip and a rigid plate based on a camera-based eccentricity control. The desired eccentricity profile was given from the mathematical relationship between the eccentricity and the surface deformation, which is obtained by known material parameters, vertical/lateral forces applied on the contact area, and static friction coefficient. The performance of the developed device was evaluated through human experiments. The experimental results show the statistically significant difference between presented weight and friction conditions.

We propose a method for estimating inherent surface color robustly against image noises from two registered images taken under different outdoor illuminations. We formulate the estimation based on maximum likelihood manner while considering both inter-pixel and intra-pixel relationships. We define inter-pixel relationship based on stochastic behavior of image noises and properties of outdoor illumination chromaticity. We rely on the spatial continuity of both surface color and illumination to define intra-pixel relationship. We also propose to maximize the estimation function in two step manner. Experimental results demonstrate the significant improvement of the proposed method in estimation accuracy compared to previous methods.

We propose a method for estimating demosaicing algorithms from image noise variance. We show that the noise variance in interpolated pixels becomes smaller than that of directly observed pixels without interpolation. Our method capitalizes on the spatial variation of image noise variance in demosaiced images to estimate the color filter array patterns and demosaicing algorithms. We verify the effectiveness of the proposed method using various images demosaiced with different demosaicing algorithms extensively.

The portability of an eye tracking system encourages us to develop a technique for estimating 3D point-of-regard. Unlike conventional methods, which estimate the position in the 2D image coordinates of the mounted camera, such a technique can represent richer gaze information of the human moving in the larger area. In this paper, we propose a method for estimating the 3D point-of-regard and a visualization technique of gaze trajectories under natural head movements for the head-mounted device. We employ visual SLAM technique to estimate head configuration and extract environmental information. Even in cases where the head moves dynamically, the proposed method could obtain 3D point-of-regard. Additionally, gaze trajectories are appropriately overlaid on the scene camera image.

In this research, a Universal map, which can be converted to individual maps for heterogeneous mobile robots, is proposed. A Universal map can be generated using our developed measurement robot, and it is composed of a textured 3D environment model. Therefore, every robot can use a Universal map as a common map, and it is utilized for various localization technologies such as view-based and LRF-based methods. In LRF-based localization, accurate localization is achieved using a specific map, which is generated from Universal map. In a view-based approach, localization and navigation are achieved using rendered images. The use of a Universal map enables generation of these maps automatically. The effectiveness of this approach is confirmed through experiments.

Generating natural motion of an articulated object with higher DOF (e.g., humanoid robot and robot hand) is a crucial issue in robotics and computer graphics fields. Use of the motion capture data is one of the solutions, but it requires expensive device and time-consuming measurement. In this paper, we propose a method for generating natural hand motion to play a piano from the inputted music score. The proposed method uses inverse kinematics while considering naturalness of hand poses. We revisit background of the inverse kinematics based on the maximum likelihood estimation and use the prior model of the hand pose to achieve the naturalness. We evaluate the effectiveness of the proposed method using voluntary survey.

Improvements in hand pose estimation, made possible by refining the model matching step, is necessary in creating a more natural human-robot interface. Individualizing the 3D hand model of the user can result to a better hand pose estimation. This paper presents a way to accomplish the individualization by estimating the length of the finger links (bones), which is unique for every user. The 3D model of the hand is made up of voxel data derived from silhouette images obtained by multiple cameras and the finger link is estimated by searching a set of models generated from the calibration motion of the fingers. Initial pose estimation result using the model shows the feasibility of the system.

In this paper, we focus on the hand posture when a human subject pinches an object, and the correlation between the results of sensory evaluation by the subject and the tendon force of the subject's fingers are investigated. Surface electromyogram (surface EMG) and pinching force are measured during the pinching motion of human subjects. Experimental results show that subjects feel comfortable pinching a 60 [mm] cylinder. The surface EMGs are lower in the vicinity of 60 [mm]. Furthermore, a tendon-driven robot hand is developed as a sensing hand prototype. The surface EMGs and the motor torque are compared in a pinching experiment using the tendon-driven robot hand. Experimental results show that the tendon-driven robot hand is effective for quantitatively evaluating pinching effort. Simulation of the pinching motion is conducted using a simple finger model. The simulation results show the importance of finger posture and the moment arm in estimating tendon force. Taken together, these results demonstrate the possibility of conducting quantitative evaluation using the tendon-driven robot hand.

In human-robot interaction, it is important for the robot to know the head movement, gaze direction and expression of the conversation partner, since such information are deeply related with the attention, intention and emotion. Recently, many types of real-time measurement systems for head pose and gaze direction have been proposed and utilized for human interfaces and ergonomics applications. We have proposed a face measurement system which is non-contact and passive by utilizing a stereo camera pair. However the necessity of the use of stereo camera has several drawbacks. To cope with this problem, this paper proposes a method to measure the 6 DOF motions of the head by utilizing a single camera. The use of 3D face model enables us to predict six 2D motion vector fields of facial features which correspond to six unit motions in translation and rotation. An actual 2D motion vector field obtained by tracking facial features is then resolved into a linear combination of the predicted six motion vector fields. The estimated 6 DOF motions can be obtained as the six coefficients of the linear combination. Through the evaluation of the implemented motion estimation method, it was shown that the accuracy of the proposed estimation method using a single camera was comparable with that of the conventional method using a stereo camera pair. Finally developed face tracking system is demonstrated by a humanoid robot interacting with two persons, which takes advantage of the wide field of view of the camera system due to the unnecessity of the stereoscopy.

This paper demonstrates that a 1-DOF planar ball-throwing robot has the capability of controlling three kinematic variables of a ball independently: translational velocity, angular velocity, and direction. The throwing motion is modeled using two underactuated contact dynamics, called a finger-link contact model and a fingertip contact model, with a unidirectional transition from one model to another. A combination of a preliminary global search method and a search algorithm based on a simulated annealing (SA) algorithm provides joint torque commands for this highly nonlinear system. An experimental system with a 1-DOF planer manipulator has been developed that throws a disk (ball) in a frictionless plane. The experimental results confirm the validity of the contact models and the feasibility of independent control of the three kinematic variables.

Continuous action sets are used in many reinforcement learning (RL) applications in robot control since the control input is continuous. However, discrete action sets also have the advantages of ease of implementation and compatibility with some sophisticated RL methods, such as the Dyna [1]. However, one of the problem is the absence of general principles on designing a discrete action set for robot control in higher dimensional input space. In this paper, we propose to construct a discrete action set given a set of basis functions (BFs). We designed the action set so that the size of the set is proportional to the number of the BFs. This method can exploit the function approximator's nature, that is, in practical RL applications, the number of BFs does not increase exponentially with the dimension of the state space (e.g. [2]). Thus, the size of the proposed action set does not increase exponentially with the dimension of the input space. We apply an RL with the proposed action set to a robot navigation task and a crawling and a jumping tasks. The simulation results demonstrate that the proposed action set has the advantages of improved learning speed, and better ability to acquire performance, compared to a conventional discrete action set.

In this paper, we propose a tendon skeletal finger model and discuss which finger postures the human feels easy to pinch based on the tendon forces and the human experimental results. The finger model mimics a human tendon skeletal structure. The tendon forces during the pinching motion were simulated using the finger model. Simulation results show that the tendon forces closely mirror the human muscle activity. Sensory evaluation of subjective pinching effort was conducted with five subjects. The subject pinched five kinds of cylinders, from 20 [mm] to 100 [mm]. The pinching force and the surface EMGs were simultaneously measured in the experiment. Based on the human questionnaire tests, we investigated which finger postures the human feels easy to pinch a cylinder. The results show that the pattern of the EMGs measured by the experiment is very similar to that of the tendon forces calculated by the finger model simulation. This indicates that the tendon force is a useful index of the subjective pinching effort and it can be used for the quantitative evaluation instead of EMGs.

Humanoid robots and robotic manipulators with good dexterity are promising to enhance the factory productivity in next generation. Dexterity of robotic manipulators can be achieved by equipping an anthropomorphic robot hand with multiple fingers. Additionally, in order to manipulate various tools that humans are using in daily life, the size and the exerting force of the robot hand should be similar to that humans are. In this paper, we proposed a human-sized multi-fingered robot hand with detachable mechanism at the wrist. The robot hand can be split at the wrist into the hand part and the actuator part. The fingers are driven by wires and are controlled by actuators embedded in the arm part. The driving force from the arm part is transmitted to the hand part by gear mechanism at the wrist. The developed robot hand has the size of 200[mm](length) × 78[mm](width) × 24.6[mm](thickness) and can exert 10[N] at the fingertip. The performance of the developed robot hand was shown by a motion control experiment.

We propose a view-based indoor/outdoor navigation method as an extension of the view-sequence navigation. The original view-sequence navigation method uses the template matching method with normalized correlation for localization. Because the matching method is sensitive to local illumination changes, it is only used for indoor environment. In this paper, we propose to adopt the accumulated block matching method to improve robustness against locally changing illumination, in which a template is split into small patches and matched by maximizing the average of the normalized correlations of all the patches.We also propose a localization criterion which helps the robot decide its motion. Our experimental results demonstrate that the proposed methods can be applied to both indoor and outdoor environments.

This paper presents the PAS-Arm (Passive ASsist Arm), a novel assist arm based on passive robotics. PAS-Arms are intended for direct physical interaction with a human operator. PAS-Arms are physically passive. Their purpose is not to enhance human strength but to produce specified but arbitrary two-dimensional guiding surfaces that constrain and guide the motion of the human operator. PAS-Arms have three joints and a three dimensional workspace but possess only two degrees of freedom due to the reduction of degrees of freedom created by a combination of continuously variable transmissions (CVTs) and differential gears. We have built a CVT test apparatus and developed a prototype of the PAS-Arm. We discuss the design of the CVT and the experimental results of the CVT test apparatus. The experimental results of the prototype are then presented.

This paper presents a new continuous action space for reinforcement learning (RL) with the wire-fitting. The wire-fitting has a desirable feature to be used with action value function based RL algorithms. However, the wire-fitting becomes unstable caused by changing the parameters of actions. Furthermore, the acquired behavior highly depend on the initial values of the parameters. The proposed action space is expanded from the DCOB, proposed by Yamaguchi et al., where the discrete action set is generated from given basis functions. Based on the DCOB, we apply some constraints to the parameters in order to obtain stability. Furthermore, we also describe a proper way to initialize the parameters. The simulation results demonstrate that the proposed method outperforms the wire-fitting. On the other hand, the resulting performance of the proposed method is the same as, or inferior to the DCOB. This paper also discuss about this result.

This paper presents the haptic interface system that consists of a base object and a haptic device. The desired force response is achieved by the combination of the real force response of the base object and the virtual force exerted by the haptic device. The proposed haptic augmented reality (AR) system can easily generate the force response of a visco-elastic object with a cheap haptic device and a base object that has the similar visco-elastic property to the target object. In the demonstration, the force response of the target object was generated by using a haptic device only (VR) and using both a haptic device and a base object (AR), respectively. The evaluation experiments by participants show that the AR method has better performance than the VR method. This result indicates the potential of the proposed haptic AR interface.

In this paper, we propose a method to estimate the group attention that is the focus of attention of multiple people. It utilizes the face direction as 3D vectors and estimates the position of group attention defined as the intersection of the multiple vectors. As a result, the position of the group attention can be represented as an arbitrary 3D position unlike other research in which only registered objects can be the focus of attention. As experiments, the group attention at word-chain game is estimated and the feasibility of the method is confirmed. We applied the proposed method to video conferencing, and developed the attention tracking system which acquires the group attention in real-time.

This paper addresses pose estimation of a hand in motion through a model-based vision-based system. Previous research on human hand tracking and pose estimation usually suffers from limited number of degrees-of-freedom estimated, camera orientation (i.e., the hand is restricted to a particular pose while moving) and occlusion. We describe and evaluate a system that allows several DOF of the hand to be estimated without hindering its motion and while minimizing occlusion. To allow for a complete 3D motion, a voxel model and a skeletal model of the hand are used. The system uses multiple viewpoint cameras to obtain information of the hand motion. Due to the non-linear characteristics of the system, Unscented Kalman Filter (UKF) is used to track the hand motion. UKF estimates the hand pose by minimizing the difference between the skeletal model and the voxel model. Estimation results from different hand motions of up to 15DOF show the feasibility of the system.

This paper reports a method that uses humanoid robots as a communication medium. There are many interactive robots under development, but due to their limited perception, their interactivity is still far poorer than that of humans. Our approach in this paper is to limit robots' purpose to a non-interactive medium and to look for a way to attract people's interest in the information that robots convey. We propose using robots as a passive-social medium, in which multiple robots converse with each other. We conducted a field experiment at a train station for eight days to investigate the effects of a passive-social medium.

The aim of this paper is to arbitrarily modify the load force of any selected muscle by using an exoskeleton, thus enabling "pinpointed" motion support, rehabilitation, and training. An advanced dynamic model, called a musculoskeletal-exoskeletal integrated human model, is developed. The driving-forces of the pneumatic actuators are designated by a muscle force control algorithm using the integrated model. A prototype power-assisting system is developed using pneumatic rubber actuators. The feasibility of the muscle force control is analyzed from a view point of nonlinear programming. The validity of the method is confirmed by measuring surface electromyographic signals of related muscles.

This paper proposes a multi-modal telecommunication system using a facial expression robot. We developed a telecommunication system which projects the facial expression of an operator to a remote place using the facial expression robot "Infanoid2." The facial information of the operator is measured using a stereo camera system and projected through a robot in order to communicate with another person in a remote location. Impression evaluation experiment is performed using this system. This paper discusses the effectiveness of robots as a telecommunication medium based on the experimental results.

This paper presents a simple task instruction scheme for a supporting robot in a daily life environment. It is difficult for a robot to perform an automatic task in the daily life environment because its sensors are not perfect in such an environment and the world model of the robot cannot cover the specific details of this environment. On the other hand, a user of the robot may control the robot by full teleoperation scheme. But the teleoperation puts the burden on the user. Instead, we propose a new instruction scheme for the supporting robot in which the user selects a task model which the robot can perform and indicates the key points on the monitoring image of the robot task environment according to the robot's request. The monitoring image is captured by a stereo camera and the range image is stored simultaneously. Therefore the system can detect the important position parameters of the target object with the indication provided by the user and the robot can complete the task with the information which the user simply gives the system.

Intelligent robots will make a chance for us to use a computer in our daily life. We implemented a humanoid robot for the computerized university guidance at first and then some capabilities for the natural interaction are added. This paper describes the hardware and software system of this humanoid with interaction ability. HRP-2 sitting opposite to a user across a table can detect gaze direction, head pose and gestures using a stereo camera system attached to the head. In addition, our system can recognize the user's question utterance and non-stationary noise such as coughing and sneezing using microphones. Using efficiently such information, HRP-2 can answer the question by its synthesized voice with gestures and do tasks such as passing objects on the table

Controlling devices using physiological signals is an increasingly active research field in human-robot interaction. Using brain signals, or Electroencephalograms (EEG), to control devices is widely known as Brain-Computer Interfacing (BCI). In a BCI system, brain signals are used to directly control devices and not as a communication channel, so target applications are restricted to handicapped and aged people. On the other hand, it is possible for healthy people to simply control devices using voice and body movements. In this paper, quantifying human's state of mind and emotions is investigated by analyzing changes in brain activity. The existence of a relation between robots' bodily expressions and their impressions on the observer is demonstrated. We also show a correlation between the nature of the expression and the power of the low-alpha channel in brain signals. Robots can make use of this quantification as an extra channel of information to achieve smoother and more human-like adaptive interaction

In this research, we propose a method for generating templates used for manipulation recognition that considers familiarization and fatigue. Our recognition system allows quantitative comparison of similarities among manipulations performed by observing contact state transitions on a palm surface. The system detects the contact states on the palm surface using an attached tactile sensor sheet on a manipulated object. In experiments, familiarization and fatigue are detected by measuring the variance of the manipulation. Experimental results indicate that manipulation can be divided into three periods: learning, familiarization, and fatigue. The results also indicate that variation of the manipulation becomes small when the template is generated by manipulation data during the familiarization period

During an interaction process, people usually adapt their behavior according to the interpretation of their partner's bodily expressions. It is not known how much similar expressions performed by robots affect a human observer. This paper explores this issue. The study shows a correlation between the nature of the bodily expressions, through the result of questionnaires, and the effect on brain activity. It has been demonstrated that unpleasant bodily expressions of the robot elicit unpleasant impressions and vice versa. This was observed through brain activity in a specific area when the expression is pleasant, and in another area when it is unpleasant.

This paper describes a method of measuring the incipient slip of the sole in order to clarify the relation between a plantar tactile sensibility and a stabilization mechanism for a slip. The incipient slip measuring device for walking analysis by using high-speed cameras and acceleration sensor is developed. The measurement device can be measured the plantar skin deformation during a walk. The incipient slip is confirmed by using the developed device.

In this paper, the CPG-based control method for the rotating manipulation that can adaptively change the issuing cycle of motion-triggers is proposed. The CPG model consists of the mutual inhibition network and the rotating manipulation is performed by the motion-triggers that are issued from the CPG model. The smooth change of the desired velocity can be given by the output of neurons that keep smooth increase and decrease. Experimental results using 4-fingered hand system suggest that the issuing cycle of the motion-triggers can be adaptively changed by the feedback of joint angle according to an object size.

This paper introduces a multifingered robotic hand eNAIST-Handf and a grip force control by slip margin feedback. The developed prototype finger of the NAIST-hand has a new mechanism by which all 3 motors can be placed inside the palm without using wire-driven mechanisms. A method of grip force control is proposed using an incipient slip estimation. A new tactile sensor is designed to active the proposed control method by the NAIST-Hand. This sensor consists of a transparent semispherical gel, an embedded small camera, and a force sensor in order to implement the direct slip margin estimation. The structure and the principle of sensing are described.

In this paper, a novel pointing device is proposed that utilizes the deformation of the fingertip. When a fingertip is pressed and slightly slid on a rigid plate, a partial slip, called an "incipient slip", occurs on the contact surface. While the deformation around the center of the contact area is small during the incipient slip, the boundary region moves to the sliding direction of the fingertip. The deformation changes depending on the sliding distance of the fingertip and the exerted force on the contact surface. The velocity of the pointer can be determined by the estimated distance and force based on the measurement of the deformation. In this study, the correlation between the sliding distance of the fingertip and the deformation and between the exerted force and the deformation are investigated. The degree of the deformation due to the sliding motion can be estimated based on the detected fingerprint center. The group delay spectrum tracking method is proposed for the detection of the fingerprint center. A prototype pointing device is developed to evaluate the operationality of the proposed device. Comparative experiments with conventional pointing devices are conducted. The validity of the proposed device is confirmed by the experiments.

In this paper, a grip-force control of an elastic object is proposed based on a visual slip-margin feedback. When an elastic object is pressed and slid slightly on a rigid plate, a partial slip, called "incipient slip," occurs on the contact surface. The slip margin between an elastic object and a rigid plate is estimated based on the analytic solution of a Hertzian contact model. A one-degree-of-freedom gripper consisting of a camera and a force sensor is developed. The slip margin can be estimated from the tangential force measured by a force sensor, the deformation of the elastic object and the radius on the contact area both measured by a camera. In the proposed method, the friction coefficient is not explicitly needed. The "eccentricity" is used to estimate the displacement of the elastic object at the contact area with high accuracy. The grip force is controlled by a direct feedback of the estimated slip margin. The proof of the contact stability by the proposed control is analytically given. As a result, the slip margin is maintained at a desired value, without occurring the gross slip against a disturbance traction force to the object. The validity of the proposed method is confirmed by experiments.

A number of researches for the dexterous manipulation have been studied in robotics and physiologic fields independently. Multi-fingered manipulation for dexterous robot hands should be based on analysis of human skills. The purpose of this study is to apply CPG-based control to the dexterous manipulation. In this paper, CPG-based manipulation is proposed for the rotating manipulation of the dexterous hands. Finger manipulation patterns are investigated when a subject rotates a cylindrical object. Typical contact patterns during rotating manipulation are measured by FSRs. Based on the observation, the neural circuit model which generates a similar contact pattern is constructed. The experimental results by the computer simulation suggest that rotating manipulation can be performed using the constructed CPG model.

In this paper, we describe a robotic wheelchair system as guide robot To build a smart interface for a guide robot, it is important for the system to know the user's attention. This system can detect gaze direction of the user using a real-time stereo vision system, and can recognize its position within the surrounding environment using a range sensor and a map. Based on the detection of both the gaze direction and the position within the map, the robotic wheelchair can estimate the user's fixation, from which the position where the user is paying attention is estimated. We adopt an attention representation based on histogram. The frequency of fixation is represented by an "attention histogram". Experimental results indicate the validity of the control assistance to change the route of the autonomous run using the estimation of user's attention.

In this paper, a grip-force control of an elastic object is proposed based on a visual slip margin feedback. When an elastic object is pressed and slid slightly on a rigid plate, a partial slip, called "incipient slip" occurs on the contact surface. The slip margin between an elastic object and a rigid plate is estimated based on the analytic solution of Hertzian contact model. A 1 DOF gripper consists of a camera and a force sensor is developed. The slip margin can be estimated from the tangential force measured by a force sensor, the deformation of the elastic object and the radius on the contact area both measured by a camera. In the proposed method, the friction coefficient is not explicitly needed. The grip force is controlled by a direct feedback of the estimated slip margin, whose stability is analytically guaranteed. As a result, the slip margin is maintained to a desired value without occurring the gross slip against a disturbance load force to the object.

A direct teaching method is effective for multi-fingered robot hand designed similar to human's hand structure. To achieve an instinctive teaching, it is necessary to perceive manipulation patterns by observing a human hand. In this paper, a perception system of human manipulation is developed as a part of direct teaching system for multi-fingered robot hand. Conventional perception systems have difficulties in perceiving complex manipulations obtained by dextrous finger actions due to the low accuracy of the CyberGlove. This paper proposes an operation measurement system using an object that is specially designed for teaching. This object consists of a six-degree-of-freedom position/orientation sensor and a tactile sensor sheet. This paper also proposes a method of spotting perception by observing a change of the contact state of the palm surface. The effectiveness of this method is confirmed by experiments.

In this paper, the global RAC-measure is proposed to a redundant manipulator mounted on a flexible body for a task-space feedback control. The tip control should be carefully designed when the dynamics of the manipulator and the flexible body are coupled such as humanoids. The measure is used to determine the redundant degrees of freedom of the manipulator based on the RAC (robust arm configuration) concept. The proposed measure evaluates the global robustness of arm configuration by mixing each RAC-measure for individual vibration mode. The redundancy of the manipulator is determined for a desired tip trajectory based on the proposed measure. As a result, a high-gain tip position control can be applied without considering the base flexibility. The validity is confirmed by a feedback-based continuous tracking for a 4 DOF human-like redundant manipulator mounted on a 3 DOF flexible body.

In this paper, we introduce an annotation-based assistance system for an unmanned helicopter with a wearable augmented reality environment. In this system, an operator controls the helicopter remotely while watching an annotated view from the helicopter through a head mounted display (HMD) with a laptop PC in a backpack. Annotations assist the operation indicating some conditions of the helicopter and a name of buildings nearby. The position and the attitude of the helicopter is measured by GPS and a gyroscope, and sent to the operator's PC via a wireless LAN.

In this paper, we study the relation between the user's degree of understanding and his/her eye movements; in an effort to realize a proactive interface that monitors the user and provides a contextual support. The application is a gaze sensitive dictionary that helps the user when reading a text in a browser's window. Not only is the user's gaze analyzed but also the context and thus the difficulty degree of the text being read. The experiment results suggest using regressions as an indicator to trigger the help process along with a context grounding approach.

In this paper, a novel pointing device is proposed which utilizes the deformation of the fingerprint. When the fingertip is pressed and slid slightly on a rigid plate, a partial slip, called "incipient slip", occurs on the contact surface. Since the deformation around the center of the contact surface is small, the center of the fingerprint can be detected during the incipient slip. The group delay spectrum (GDS) tracking is utilized to detect the center of the fingerprint. The velocity of the pointer is controlled by the changes of the contact area on the basis of the tracked center. A prototype pointing device is developed to examine the operationality of the proposed method. The experimental results show that an operator can control the pointer with high accuracy.

In this paper, the role of the human being toe is investigated. The change of the sole pressure distribution using a sole pressure distribution measurement system and that of the toe pressure using a toe pressure measurement system at walk by the existence of a toe are investigated. In order to investigate the change of the whole walking, the affection of a walk posture using a 3D motion capture system by the existence of a toe is also measured. Furthermore, the change of the amount of energy consumption at walk by the existence of a toe is measured by a spirometry system. At last, we investigate the roles of the toe, such as, the efficiency of the big and 2nd toe, and the stabilization of the posture, by the synthesis of these results.

Manipulator tasks such as assembly and disassembly can generally be divided into several motion primitives. We call these "skills" and explain how most manipulator tasks can be composed of skill sequences. Skills are also used to compensate for errors both in the geometric model and in manipulator motions. There are dispensable data in the shapes, positions and orientations of objects when achieving skill motions in a task Therefore, we can derive simplified geometric models by considering both dispensable and indispensable data in a skill motion. We call such robust and simplified models "false models." This paper describes our definition of false models in planning and visual sensing and shows the effectiveness of our method using examples.

The conventional reinforcement learning method has problems in applying to real robot tasks, because such method must be able to represent the values in terms of infinitely many states and action pairs. In order to represent an action value function continuously, a function approximation method is usually applied. In our previous work (2000), we pointed out that this type of learning method potentially has a discontinuity problem of optimal actions for a given state. In this paper, we propose a method for estimating where a discontinuity of the optimal action takes place and for refining a state space incrementally. We call this method an continuous valued Q-learning method. To show the validity of our method, we apply the method to a simulated robot.

Proposes a method for hand pose estimation that can be used for 3D free-form input interfaces. The aim of the method is to estimate all joint angles to manipulate objects in the virtual space. In this method, the hand regions are extracted from multiple images obtained by the multi-viewpoint camera system. By integrating multi-viewpoint silhouette images, a hand pose is reconstructed as a "voxel model". Then all joint angles are estimated using three dimensional model fitting between hand model and voxel model. Two experiments were performed: (1) the estimation of joint angles by the silhouette images from the hand-pose simulator, (2) hand pose estimation using real hand images. The experimental results indicate the feasibility of the proposed algorithm for vision-based interfaces, though it requires faster implementation for real-time processing.

To build smart human interfaces, it is necessary for a system to know a user's intention and point of attention. Since the motion of a person's head pose and gaze direction are closely related to his/her intention and attention, detection of such information can be utilized to build natural and intuitive interfaces. In this paper, we describe a behavior recognition system based on the real-time stereo face tracking and gaze detection system to measure head pose and gaze direction simultaneously. The key aspect of our system is the use of real-time stereo vision together with a simple algorithm which is suitable for real-time processing. Our system we can significantly simplify the algorithm for 3D model fitting to obtain the full 3D pose of the head compared with conventional systems that use monocular camera. The recognition of attentions and gestures of a person is demonstrated in the experiments.

In the multi-agent environment, it is important to identify position and orientation of multiple robots for accomplishing a given task in cooperative manner. This paper proposes a method for estimating position and orientation of multiple robots using multiple omnidirectional images based on geometrical constraints. Our method reconstruct not only the relative configuration between robots but also an absolute one using the knowledge of landmarks in the environment. Even if there are some obstacles in the environment, our method can estimate absolute configuration between robots based on the results of self-localization of each robot.

We propose a new architecture for recognizing objects based on a concept of "embodiment" as one of primitive functions for a cognitive robot. We define the term "embodiment" as the size and shape of the agent's body, locomotive ability and its sensor. According to embodiment, an object is represented by reaching action paths, which correspond to a set of sequences of movements taken by the agent for reaching the object. Visual information is used to obtain sensorimotor mapping which represents the relationship between the change of object's appearance and the movement of the agent. On the other hands, tactile information is utilized to evaluate the change of physical condition of the object caused by such movement. By means of this method, the agent can recognize an object without depending on its position and orientation in the environment. The experimental result of computer simulation is shown to validate the method.

In this paper, we propose a new strategy for detecting the joint among two potsherds. Joint detection problems were studied in jigsaw puzzle assembling. However, the shape assumptions of a piece used in the past researches cannot be applied to joint detection to reconstruct broken earthenware. To detect the joint, the most similar section among two contours must be detected by partial verification. In our strategy, each contour is segmented for partial verification without making any assumption about the shape of a potsherd, unlike previous jigsaw puzzle assembling methods. Our strategy consists of five processes: the contour segmentation process, the segment description process, the segment verification process, the candidate extraction process, and the candidate verification process. We also present experimental results of our strategy with two-dimensional images of potsherds.

Contact state information is indispensable for performing skillful tasks in which an end-effector should be in contact with the environment. In previous work, the authors introduced the concept of a pseudo contact point that is useful for tasks involving contact as one expression of the contact state. In this paper, an extended definition of the pseudo contact point is proposed. The pseudo contact point is defined as the foot of the perpendicular from the nearest point on a reference object to the line on which the contact point and external force vector lie. The system may estimate the contact state and its transitions by monitoring the pseudo contact point in spite of the fact that the pseudo contact point does not always denote the real contact point. Experimental results suggest the effectiveness of our method, which is particularly valuable in providing an intuitive positional description of a contact state.

In order to keep visual tracking systems with color segmentation technique running in a real environment, an online learning method to update models for adapting them to dynamic changes of surroundings needs to be developed. To deal with this problem, we propose an online visual learning method for color image segmentation and object tracking in a dynamic environment. Our method utilizes a fuzzy ART model which is a kind of neural network for competitive learning. The mechanism of this neural network is suitable for online learning and is different from that of a backpropagation type neural network. In order to use the fuzzy ART model for coder segmentation online, we transform the color signal that the framegrabber used yields to a particular color space called Yr/spl theta/ space. To show the validity of our method, we present some results of experiments using sequences of real images.

The paper describes an integrated teleoperation system for maintenance tasks with planning functions based on manipulation skills. We embed planning functions into a telerobotics system to make the system more flexible and robust. A motion teaching system based on contact state transition, a geometric modeling system using teaching trees, and a task execution system based on manipulation skills are integrated. The design concept of the system and essential technologies are described. An experimental task is explained to demonstrate the efficiency of the telerobotics system.

The authors studied the planning for wire obstacle avoidance by manipulators. It is difficult to know the location and direction of an obstacle with an ultrasonic sensor, since distance is the only information provided by the sensor. The authors propose here a method for estimating the location and direction of a wire obstacle by using multiple items of information on distance. The authors report the results of simulation of path modification by detecting wire obstacles based on the proposed method.

Insertion of a flexible wire into a hole is a difficult task because the wire has a tendency to buckle under external forces. Moreover, the plastic deformation occurs on a flexible wire when the force acting on the wire is big. In this paper, we propose a method of inserting a flexible wire whereby a force acts on the wire and the shape of the wire is observed by stereo vision. A strategy to transform the deformed wire to the straight one and to insert the wire into a hole is presented. Finally, some experimental results are shown.

Two decentralized cooperation controllers are presented for trajectory tracking of two manipulators handling an object. The controllers control positions of the robots distributively by using trajectory errors of the object in the task space. In the first controller, the internal force between the object and the manipulator is controlled only by feedforward of the desired force. The second controller uses a force feedback. No communication is required between the manipulators in the both controllers. Their globally and exponentially asymptotic stability are guaranteed by Lyapunov functions.

Measuring the force on a flexible wire is difficult by using force/torque sensors when the flexural rigidity of the wire is small because the wire buckles and the force on the wire is smaller than the force that can be measured by a force/torque sensor. This paper presents a method of calculating the force on a wire from its shape observed by stereo vision. We propose a method of inserting the wire into a hole in a wall by using the estimated force. Three experiments have been successfully carried out.

Monitoring of a contact state is an important technique to achieve tasks in which an end-effector should be in contact with an object. This paper introduces a pseudo contact point that is useful for precise tasks including contact processes. When the position of a point contact between the end-effector and the object moves to another position, the pseudo contact point denotes the present contact point. The pseudo contact point is defined as the foot of the perpendicular from the previous contact point to the line on which the external force vector lies. Experimental results suggest the applicability of our method.

Generally, a manipulator task can be divided into several motion primitives called "skills". Skill-based motion planning is an effective way to execute a complicated task. Also when planning tasks, a technique of fine motion planning such as the backprojection method in configuration space is often used. In this paper we introduce the concept of skill into the backprojection method. A backprojection is shaped by a pattern of motion on each skill primitive, and a skill library is constructed. Motion planning based on backprojection is achieved by combining several skills in the skill library. Therefore, a general and skilful motion planning can be derived.

In robot motion planning, geometric modeling plays an important role. Generally the shapes of objects such as factory products can be stored in a computer software database, so the elements that need to be decided are positions and orientations. In this paper surface-based geometric modeling using a task-oriented teaching tree is proposed. In this modeling, combinations of surfaces for deciding positions and orientations of objects are represented in a depth-first tree based on a kind of task. Therefore, the operator can choose easily one combination out of several compared with the obtained data. Moreover a geometric model of the object suited to the manipulation task can be obtained.

Geometric modeling of the environment is important in robot motion planning. Generally, shapes can be stored in a database, so the elements that need to be decided are positions and orientations. In this paper surface-based geometric modeling using a teaching tree is proposed. In this modeling method, combinations of surfaces are considered in order to decide the positions and orientations of the object. The combinations are represented by a depth-first tree, which makes it easy for the operator to select one combination out of several. This method is effective not only in the case when perfect data can be obtained but also when conditions for measurement of three-dimensional data are unfavorable which often is the case in the environment of a working robot.