This paper proposes a novel visual Learning from Demonstration (LfD) system that teaches robots to learn tasks in which an object is put into another object, stacked, or transported by a tool. In such tasks, observation targets become invisible owing to occlusion or frame-out. The proposed “Visual Hierarchy-based Function Estimator (Hi-Fes)” is inspired by the knowledge derived from the field of psychology that uses the visual hierarchy relationships to estimate the changes in observation targets. Hi-Fes employs a mechanism to interpolate the features when the targets are unrecognizable and state variables are incalculable directly. This method facilitated visual learning of complex state changes between multiple targets, in which the target becomes invisible or has a long period of invisibility, difficult for conventional learning methods. The proposed method was implemented in a life-sized humanoid robot and was evaluated in learning based on demonstration experiments. The results demonstrated the effectiveness of our approach.

We propose a method for a robot to manipulate an unknown door based on a single user instruction. The primary contributions of this paper are (i) to reduce the user instruction to a single click and (ii) to develop an efficient method to estimate an appropriate shape and manipulation model for a target door by integrating visual and touch information obtained by a robot. The proposed method first detects door candidates using a 3-D camera and then estimates the manipulation model of each candidate based on prior learning results. During door manipulation, the system integrates visual and touch information to estimate the shape and manipulation model to generate an appropriate motion. We evaluated the proposed method experimentally, and the results prove that the proposed method is effective.

This paper describes grasp point selection on an item of clothing randomly placed on a table. The input data for our proposed method is a range image captured from a fixed, 3D range camera. Hem elements are extracted from the data, and their relationships are characterized for both similarity measures and grasp point evaluation. Experiments using real images, targeting a piece of clothing, show the effectiveness of the proposed method.

This paper presents a method for daily assistive robots to manipulate objects whose contact relationships are important. In our approach, contact relationships between multiple objects such as “on” and “into” are estimated using the hierarchical relationships between the objects' regions in the images. The hierarchical states are used for failure recovery processes so that the target contact state is achieved. We tested our method in two daily tasks by HRP-2VZ humanoid robot: filing, and serving an egg with a spatula. These results suggested the effectiveness of our approach.

This paper describes about a method of clothing classification using a single image. The method assumes to be used for building autonomous systems, with the purpose of recognizing day-to-day clothing thrown casually. A set of Gabor filters is applied to an input image, and then several image features that are invariant to translation, rotation and scale are generated. In this paper, we propose the descriptions of the features with focusing on clothing fabrics, wrinkles and cloth overlaps. Experiments of state description and classification using real clothing show the effectiveness of the proposed method.

Many countries around the world face three major issues associated with their aging societies: a declining population, an increasing proportion of seniors, and an increasing number of single-person households. To explore assistive technologies that can help solve the problems faced by aging societies, we have tested several information and robot technologies. This paper introduces research on a home-assistant robot, which improves the ease and productivity of home activities. For people who work hard outside the home, the assistant robot performs chores in their home environment while they are away. A case study of a life-sized robot with a humanlike functional body performing daily chores is presented. An integrated software system incorporating modeling, recognition, and manipulation skills, as well as a motion generation approach based on the software system, is explained. Moreover, because housekeepers perform chores one after another in their daily environment, we also aim to develop a system for continuously performing a series of tasks by including failure detection and recovery.

This paper describes a software system integration of daily assistive robots. Several tasks related to cleaning and tidying up rooms are focused on. Recognition and motion generation functions needed to perform daily assistance are developed, and these functions are used to design various behaviors involved in daily assistance. In our approach, the robot behaviours are divided into simple units which consist of 3 functions as check/plan/do, it provides us with high reusable and flexible development environment. Because sequential task execution can be achieved only after functions about failure detection and recovery, we also try to implement such functions in keeping with this approach. In addition to using simple behavior unit, multilayer error handling is effective. Experiments doing several daily tasks with handling daily tools showed the effectiveness of our system.

This paper describes a novel method of path following for wheeled mobile robots. A number of virtual manipulators mounted on the robot are used to make the robot poses which satisfy both of path following and collision avoidance. This method can calculate robot poses by using a single jacobi matrix despite of coping with several manipulators. We also present about the method of motion stabilization by using a singularity-avoiding criteria. Effectiveness of the proposed method is proven by means of simulation in both cases of omnidirectional wheelbase and 2 wheeled mobile platform.

This paper describes a navigation system in a dynamic environment for a two-wheeled inverted pendulum mobile robot, PMR. Our system is organized by localization, detection and tracking of pedestrians, and trajectory planner. The localization is robust to effects of moving obstacles and pitching movements of the robot, and the trajectory planner creates a path with a certain smoothness considering movements of pedestrians. In addition, the planner introduces strategies to avoid pedestrians to be friendly to pedestrians around the robot. Besides, our system can run on two laptop PCs in real time. Finally, we show experimental results as well.

This paper introduces a planning method for handling furniture which exists in real world. We propose a method which is easily expandable its handle able furniture such as closet, shelf and so on. If the robot can handle such furniture autonomously, it is expected that multiple daily tasks, for example, storing a small object in a drawer, can be achieved by the robot. Because perplexing processes is needed to give the knowledge of furniture handling to the robot manually, we propose direct teaching based approach which can easily give not only how to handle the furniture but also an appearance and 3D shape of it. Combining general knowledge given in advance and manipulation procedure instructed by human directly, the robot acquires how to manipulate the storing places. The performance of the proposed method is illustrated by experiments.

This paper describes a grasp planning for a mobile manipulator which works in real environment. Mobile robot studies up to now that manipulate an object in real world practically use ID tag on an object or an object model which is given to the robot in advance. The authors aim to develop a mobile manipulator that can acquire an object model through video images and can manipulate the object. In this approach, the robot can manipulate an unknown object autonomously. A grasp planning proposed in this paper can find a stable grasp pose from the automatically generated model which contains redundant data and the shape error of the object. Experiments show the effectiveness of the proposed method

This paper describes motion planning for a mobile manipulator which works in real environment. Mobile manipulators that can manipulate an object was developed up to now. We assume that object grasping is not restricted to just one grasp way and robot can move freely as far as it is in its operational area of the object. This paper proposes the motion planning for object grasping under the previous explained conditions. There are two major issues to detect the robot pose. The first problem is pose error of the robot hand. The second problem is kinematical redundancy of the mobile manipulator. The pose is evaluated from the amount of joint motions for recovery of hand pose error. Experiments show the effectiveness of the proposed method.

This paper describes 3-D shape modeling performed by a mobile robot. We propose a method in which the robot in the real environment acquires the accurate shape and constructs a 3-D model of an object using Computer Vision techniques under the condition that position and shape of the object is unknown. 3-D modeling is performed from image streams which are captured by a camera placed on the robot. The results show the effectiveness of our method.

This paper describes how to acquire 3-D shape of an object and to estimate robot pose by observing an unknown object. A single camera is placed on the robot, image streams of the target object are captured by the camera on a mobile robot moving around the object. Computer vision techniques are utilized under the condition that shape, size and accurate position information of the target object is not given in advance, but the robot finds the target object and reconstructs the 3-D object shape by itself. In addition, it is necessary that the robot grasps relative pose between itself and the target object. Experimental result show the effectiveness of our method in point of robustness and accuracy for the mobile robot.