Humanoid robotic hands need to be versatile and capable of providing environmental information in order to serve as a platform for intelligent grasp control. To facilitate the design process of such hands, we present the KIT Robotic Hands. They have been designed to meet diverse application requirements through their scalability in size, actuation, sensorization and computing resources. The hands integrate a multi-modal sensor system, in-hand embedded processing capabilities, an adaptive underactuated mechanism and a continuously controllable thumb rotation to enhance dexterity. The flexibility of the design is demonstrated through two application-specific hand implementations: one is the ARMAR-7 hand, which has human hand dimensions for grasping daily objects in household tasks, the other is the ARMAR-DE hand, a larger hand designed for grasping bigger objects in decontamination tasks. We describe the design and mechatronics of the hands as well as an evaluation of the grasp success and image segmentation based on an in-hand integrated camera and onboard processing of visual data.

Intelligent, semi-autonomous prostheses take ad-vantage of combining autonomous functions and traditional myoelectric control. With the help of visual and environment sensors, intelligent prostheses achieve a level of autonomy which relieves the user from generating elaborate electromyographic (EMG) signals for grasp type and trajectory. To achieve the desired functionality, the semi-autonomous prosthesis must efficiently process the incoming environmental data at a high rate, with low power and high accuracy. In this paper, we propose Binary-LoRAX, a low-latency runtime adaptable classifier for the semi-autonomous grasping task of prosthetic hands. We offload the classification task to an efficient binary neural network accelerator which performs high-throughput XNOR operations on digital signal processing (DSP) blocks. To tailor the classifier’s performance to the current application scenario, we propose a frequency scaling approach which dynamically switches between two modes of operation, high-performance and power-saving. At high-performance, classifications are performed with a low latency of 0.45ms, high-throughput of 4999 FPS and power consumption of ∼ 2.15 W. This enables functions such as object localization and batch classification. Switching to power-saving mode, a latency of 80 ms is maintained, with up to 19% improved classifier battery-life. Our prototypes achieve a high accuracy of up to 99.82% on a 25 class problem from the YCB graspable object dataset.

We introduce a multi-functional robotic gripper equipped with a set of actions required for disassembly of electromechanical devices. The gripper consists of a robot arm with 5 degrees of freedom (DoF) for manipulation and a jaw gripper with a 1-DoF rotation joint and a 1-DoF closing joint. The system enables manipulation in 7 DoF and offers the ability to reposition objects in hand and to perform tasks that usually require bimanual systems. The sensor system of the gripper includes relative and absolute joint encoders, force and pressure sensors to provide feedback about interaction forces, a tool- mounted camera for screw detection and precise placement of the tool tip using image-based visual servoing. We present a data-driven method for estimating joint torques based on the output voltage and motor speed. Further, we provide methods for teaching disassembly actions based on human demonstration, their representation as movement primitives and execution based on sensory feedback. We provide quantitative results regarding positioning and torque estimation accuracy, disassembly success rate and qualitative results regarding the successful disassembly of hard disc drives.

We present a soft humanoid hand with in-finger integrated cameras and in-hand real-time image processing system for fast reactive grasping. Specifically, we describe an FPGA-based, in-hand integrated, embedded system for processing visual data captured by the five in-finger cameras while avoiding high bandwidth raw data streaming via the robots real-time data bus. The hardware acceleration allows fast detection and localization of objects based on finger-camera images and provides input for a grasping controller. To this end, we implement a resource-aware encoder-decoder Convolutional Neural Network (CNN) for pixel-wise object segmentation and run inference on the in-hand embedded system at 3.58 GOPS. We evaluate the system, consisting of the soft hand with in-finger vision and the in-hand FPGA-accelerated CNN in several experiments on the humanoid robot ARMAR-6. Specifically, we evaluate the overall system response time, the ability to perform precision grasps and test reactivity and reliability that are required for handover actions. We obtain an overall system response time of 154 ms for catching a falling object and obtain a success rate of 90 % reliability for the power drill handover tasks. Further, we successfully demonstrate ability of dexterous grasping and manipulation of a pencil from a cup.

Engineering humanoid robot hands with the ability to dexterously grasp objects of different sizes, shapes, mate-rial properties and weights requires sophisticated tactile sensing and intelligent controllers able to interpret sensory information and adapt contact forces with the object to achieve a stable and safe grasp. In this paper, we present a new soft humanoid hand equipped with a multimodal sensor system in each finger and a human-inspired grasp-phases controller that is able to detect the different phases of a grasping and manipulation task, adapt interaction forces with the manipulated object and balance the force distribution in both precision and power grasps based on tactile feedback. To evaluate the controller, we conducted experiments with the hand on the humanoid robot ARMAR-6 and 31 different soft and rigid everyday objects and food items with weights ranging from 4.8 g of a paper cup to 1133.8 g of a bottle, different shapes and material properties. The results show that grasping force can be reduced by 65% compared to a naive grasping approach using maximum force for grasping and manipulating both fragile objects without destruction as well as heavy objects.

We present a novel underactued humanoid five finger soft hand, the KIT Finger-Vision Soft Hand, which is equipped with cameras in the fingertips and integrates a high performance embedded system for visual processing and control. We describe the actuation mechanism of the hand and the tendon-driven soft finger design with internally routed high-bandwidth flat-flex cables. For efficient on-board parallel processing of visual data from the cameras in each fingertip, we present a hybrid embedded architecture consisting of a field programmable logic array (FPGA) and a microcontroller that allows the realization of visual object segmentation based on convolutional neural networks. We evaluate the hand design by conducting durability experiments with one finger and quantify the grasp performance in terms of grasping force, speed and grasp success. The results show that the hand exhibits a grasp force of 31.8 ± 1.2 N and a mechanical durability of the finger of more than 15.000 closing cycles. Finally, we evaluate the accuracy of visual object segmentation during the different phases of the grasping process using five different objects. Hereby, an accuracy above 90% can be achieved.

The development and control of prosthetic hands is an active research area and recently progress in mechatronics, sensor integration and innovative control has been made. However, integration of different components into a prosthetic hand remains challenging due to space constraints, the requirements regarding holistic integration and the need for a user interface. In this paper, we present the KIT prosthetic hand, a novel five-finger 3D printed hand prosthesis, with its underactuated mechanism, sensors and embedded control system. The hand mechanics is based on the underactuated TUAT/Karlsruhe mechanism with two motors actuating 10 degrees of freedom. The mechanism has been realized in 3D printing technologies to facilitate a personalization of the prosthetic hand in terms of size and kinematic parameters. The prosthesis has been designed as a 50thpercentile male hand. It integrates an advanced embedded system as well as an RGB camera in the base of the palm and a colour display in the back of the hand. Experiments indicate a finger tip force of 7.48 N to 11.82 N, a hook grasp force of 120 N and a hand closing time of ~ 1.3 s.