Groping behavior based on contact sensors is necessary for manipulation in an unknown environment. For those situations, it is effective for a robot to accumulate contact information as an environment map, and to plan the motions for executing the safe trial motion. We first propose a method of updating the occupancy grid map of the manipulation region from the contact information by introducing the contact sensor model. Using this map, we propose a method of sampling-based motion planning that enables the execution of the safe trial motion based on the criteria of feasibility and safety. To verify the effectiveness, we show the experimentally obtained results, showing that a real robot plans and executes the manipulation with groping behavior in the occluded environment.

To achieve tasks in unknown environments with high reliability, highly accurate localization during task execution is necessary for humanoid robots. In this paper, we discuss a localization system which can be applied to a humanoid robot when executing tasks in the real world. During such tasks, humanoid robots typically do not possess a referential to a constant horizontal plane which can in turn be used as part of fast and cost efficient localization methods. We solve this problem by first computing an improved odometry estimate through fusing visual odometry, feedforward commands from gait generator and orientation from inertia sensors. This estimate is used to generate a 3D point cloud from the accumulation of successive laser scans and such point cloud is then properly sliced to create a constant height horizontal virtual scan. Finally, this slice is used as an observation base and fed to a 2D SLAM method. The fusion process uses a velocity error model to achieve greater accuracy, which parameters are measured on the real robot. We evaluate our localization system in a real world task execution experiment using the JAXON robot and show how our system can be used as a practical solution for humanoid robots localization during complex tasks execution processes.

Whole-body holding manipulation is effective for carrying the handleless large object. In order to keep the object stability, the dexterous transition motion is necessary. From geometric and physical conditions of object manipulation, we propose the general method of generating the transition graph, which represents the object pose and grasp contact. By searching the path on the graph, the transition motion is planned automatically with considering the object motion and contact switching simultaneously. By generating and modifying the whole-body holding motion, the planned object motion is achieved stably. We show the effectiveness of the proposed method by the experiments, in which robot lifts up a large object with whole-body contact by the planned transition motion.

This paper describes a textureless object segmentation approach for autonomous service robots acting in human living environments. The proposed system allows a robot to effectively segment textureless objects in cluttered scenes by leveraging its manipulation capabilities. In our pipeline, the cluttered scenes are first statically segmented using state-of-the-art classification algorithm and then the interactive segmentation is deployed in order to resolve this possibly ambiguous static segmentation. In the second step the RGBD (RGB + Depth) sparse features, estimated on the RGBD point cloud from the Kinect sensor, are extracted and tracked while motion is induced into a scene. Using the resulting feature poses, the features are then assigned to their corresponding objects by means of a graph-based clustering algorithm. In the final step, we reconstruct the dense models of the objects from the previously clustered sparse RGBD features. We evaluated the approach on a set of scenes which consist of various textureless flat (e.g. box-like) and round (e.g. cylinder-like) objects and the combinations thereof.

Humanoid robots working in a household environment need 3D geometric shape models of objects for recognizing and managing them properly. In this paper, we make humanoid robots creating models by themselves with dual-arm re-grasping (Fig.1). When robots create models by themselves, they should know how and where they can grasp objects, how their hands occlude object surfaces, and when they have seen every surface on an object. In addition, to execute efficient observation with less failure, it is important to reduce the number of re-grasping. Of course when the shape of objects is unknown, it is difficult to get a sequence of grasp positions which fulfills these conditions. This determination problem of a sequence of grasp positions can be expressed through a graph search problem. To solve this graph, we propose a heuristic method for selecting the next grasp position. This proposed method can be used for creating object models when 3D shape information is updated on-line. To evaluate it, we compare the result of the re-grasping sequence from this method with the optimal sequence coming out of breadth first search which use 3D shape information. Also, we propose an observation system with dual-arm re-grasping considering the points when humanoid robots execute observation in the real world. Finally, we show the experiment results of construction of 3D shape models in the real world using the heuristic method and the observation system.

Many countries around the world face three major issues associated with their aging societies: a declining population, an increasing proportion of seniors, and an increasing number of single-person households. To explore assistive technologies that can help solve the problems faced by aging societies, we have tested several information and robot technologies. This paper introduces research on a home-assistant robot, which improves the ease and productivity of home activities. For people who work hard outside the home, the assistant robot performs chores in their home environment while they are away. A case study of a life-sized robot with a humanlike functional body performing daily chores is presented. An integrated software system incorporating modeling, recognition, and manipulation skills, as well as a motion generation approach based on the software system, is explained. Moreover, because housekeepers perform chores one after another in their daily environment, we also aim to develop a system for continuously performing a series of tasks by including failure detection and recovery.

A humanoid robot working in a household environment with people needs to localize and continuously update the locations of obstacles and manipulable objects. Achieving such system, requires strong perception method to efficiently update the frequently changing environment. We propose a method for mapping a household environment using multiple stereo and depth cameras located on the humanoid head and the environment. The method relies on colored 3D point cloud data computed from the sensors. We achieve robot localization by matching the point clouds from the robot sensor data directly with the environment sensor data. Object detection is performed using Iterative Closest Point (ICP) with a database of known point cloud models. In order to guarantee accurate object detection results, objects are only detected within the robot sensor data. Furthermore, we utilize the environment sensor data to map out of the obstacles as bounding convex hulls. We show experimental results creating a household environment map with known object labels and estimate the robot position in this map.

We propose a strategy for a robot to operate in an environment with movable obstacles using only onboard sensors, with no previous knowledge of the objects in that environment. Movable obstacles are detected using active sensing and a color range sensor, and when an obstacle is moved, the perception of the environment is reconstructed. Active sensing is defined as the classification of an object as either movable or static after the robot tries to push the object using its arm. This classification is collectively based on force sensor inputs, joint angles, and color range sensor inputs. In order to gather information from the environment, we use a color range sensor consisting of a TOF (Time of Flight) range sensor and conventional stereo cameras. Finally, we show experimental result in the environment with movable obstacles such as a table and chairs. Humanoid robot HRP-2 detects that a chair is a movable obstacle, moves the chair to clear a path to its goal, and then reaches the goal.

In this paper we propose a new method to manipulate heavy objects for a humanoid robot. In this method the manipulation strategy is determined based on on-line estimation of the operational force. We integrate these functions with a real-time controller that controls the external force and maintains full-body balance. The feature point of our work is that since a full-body control system includes switching of the manipulation strategy based on the operational force estimated on-line the system enables a humanoid robot to manipulate heavy objects as well as light objects. The effectiveness of our whole system is confirmed in our experiments, in which a humanoid robot manipulates up to 12[kg] while estimating the object's weight.

This paper describes a software system integration of daily assistive robots. Several tasks related to cleaning and tidying up rooms are focused on. Recognition and motion generation functions needed to perform daily assistance are developed, and these functions are used to design various behaviors involved in daily assistance. In our approach, the robot behaviours are divided into simple units which consist of 3 functions as check/plan/do, it provides us with high reusable and flexible development environment. Because sequential task execution can be achieved only after functions about failure detection and recovery, we also try to implement such functions in keeping with this approach. In addition to using simple behavior unit, multilayer error handling is effective. Experiments doing several daily tasks with handling daily tools showed the effectiveness of our system.