A robotic hand for social interaction should be capable of comfortable touch with humans. However, it is difficult to mount skin, tactile sensors, and driving mechanism required for human contact, especially holding hands, on a slender finger. In addition, in order to unitize the hand for easy use with any robot and maintainability, the mechanism must be contained within the small space of the fingers and palms. In this paper, we propose a human-sized five-fingered robotic hand named WARABI Hand. It is covered with multi-layored rubber skin to realize human-like soft and pleasant feel. Force sensors on each finger link detect contact with humans and adjust gripping force. We conducted experiments in which a humanoid equipped with WARABI Hand grasped forearm, held hands, and interlocked fingers with a person. The performance for object grasping was also evaluated. We demonstrated that our proposed hand is useful for interaction with humans including receiving and handing over things.

Recent progress of deep learning improved the capability of a robot to find a proper grasp of a novel object for different grasp modalities (e.g., pinch and suction). While these previous studies consider multiple modalities separately, several studies develop multi-modal grippers that can achieve simultaneous pinch and suction grasp (multi-modal grasp fusion) for more capable and stable object manipulation. However, the previous studies with these grippers restrict the situations: simple object geometry and uncluttered environments. To overcome these difficulties, we propose a system that consists of: 1) object-class-agnostic grasp modality detection; 2) object-class-agnostic instance segmentation; and 3) grasp template matching for different modalities. The key idea of our work is the introduction of instance segmentation to fuse multiple modalities regarding each instance eluding a grasp of multiple objects at once. In the experiments, we evaluated the proposed system on the real-world picking task in clutter. The experimental results show that the effectiveness of modality detection, instance segmentation, and the integrated system as a whole.

Robotic picking of folded objects such as books is required for picking various objects. As a folded object is easily unfolded, it is difficult to carry it stably and place it in a desired pose due to its dangling part. For overcoming this difficulty, we propose a trial-and-error picking system using our Suction Pinching Hand, which can push the dangling part up with pinch grasp until the object lifted with suction grasp is folded. That system utilizes proximity sensors on the hand to predict whether folding will succeed with a current hand pose and decide whether to retry with another pose. Also, proximity sensors, flex sensors and an air pressure sensor are used to deal with uncertainty of the image recognition, the hand hardware and suction grasp. We evaluate our proposed system with experiments of picking and placing folded objects. It is confirmed that our proposed system realizes picking with the ability of our Suction Pinching Hand to carry folded objects stably and place them in desired poses. It is also proved that our proposed system is robust against the uncertainty.

Recently, self-supervised approach is common for robot grasping. Although this approach improves success rate, it requires a long time to execute a number of grasp trials, and single-arm grasping is only considered. However, robots can grasp more various objects with two arms, and dual-arm robots such as humanoid robots are expected to execute dual-arm manipulation and overcome the single-arm limitation. In this paper, we introduce dual-arm grasping as another possible strategy and propose a multi-stage learning method for selective dual-arm grasping using Convolutional Neural Networks (CNN)for grasping point prediction and semantic segmentation. In the first stage, the network learns grasping points with the automatic annotation. Although a robot learns both single-arm and dual-arm grasping efficiently with the annotation, the robot may not be able to grasp it because the annotation algorithm is designed by human. Therefore, for the second stage, the robot samples various grasping points with both grasping strategies and learns how to grasp in the real world. In this stage, the robot obtains new possible grasping points and prunes unsuccessful ones for both grasping strategies through the robot experience. In the experiments in the real world, the adapted network achieved high success rate 76.7% in 90 trials. Since the network trained with no adaptation stage resulted in lower success rate 56.7%, this result also shows the network was refined with less than 250 times of grasp sampling. As an application of our method, we demonstrated that our system worked well in warehouse picking task.

Robots need to adapt themselves to various surroundings in order to achieve robust object search and grasp in unknown environments. For this adaptation, robot motions should be implemented as combination of primitive motions which are based on sensor reaction. Among various sensing methods, non contact sensing is required as a means of preventing operation failures such as pushing objects. Especially, proximity sensors have been proved effective in avoiding occlusion problems. In this paper, we first develop a gripper on which proximity sensors are mounted all around, and then calculate distance between the gripper and objects using proposed calibration method. This enables robots to recognize detailed shapes of objects surrounding the gripper. We also propose primitive motions for object search and grasp, and describe the contents of each motion. The motions are based on sensor information obtained from the gripper. We verify the effectiveness of our system through an experiment in which a real robot performs complex tasks by combination of the primitive motions.

Picking various objects in cluttered narrow space automatically is required for warehouse automation. In this space, multi-fingered robot hands have difficulty in grasping objects as objects are surrounded by obstacles. On the other hand, vacuum grippers have difficulty in grasping various objects stably. In this paper, we propose the Suction Pinching Hand, which has two underactuated fingers and one extendable and foldable suction finger whose fingertip has a suction cup. This hand can grasp objects in cluttered narrow space using the suction finger. In addition, it can grasp various objects stably using suction and pinch at the same time. The ability to grasp various objects stably of this hand is confirmed by tabletop experiments. We also propose a picking strategy using suction and pinch simultaneously in cluttered narrow space. We evaluate our proposed methods with shelf bin picking experiments. With our methods, a robot can pick various objects in cluttered narrow space.