The observation of robotic assembly tasks is required as feedback for decisions and adaption of the task execution on the current situation. A sequential Monte Carlo observation algorithm is proposed, which uses a fast and accurate collision detection algorithm as a reference model for the contacts between complex shaped parts. The main contribution of the paper is the extension of the classic random motion model in the propagation step with sampling methods known from the domain of probabilistic roadmap planning in order to increase the sample density in narrow passages of the configuration space. As a result, the observation performance can be improved and a risk of sample impoverishment reduced. Experimental validation is provided for a peg-in-hole task executed by a lightweight-robot arm equipped with joint torque sensors.

On-orbit servicing involves a new class of space missions in which a servicer spacecraft is launched into the orbit of a target spacecraft, the client. The servicer navigates to the client with the intention of manipulating it, using a robotic arm. Within this framework, this work presents a new robotic experimental facility which was recently built at the DLR to support the development and experimental validation of such orbital servicing robots. The facility allows reproducing a close-proximity scenario under realistic three-dimensional orbital dynamics conditions. Its salient features are described here, to include a fully actuated macro-micro system with multiple sensing capabilities, and analyses on its performance including the amount of space environment volume that can be simulated.

This paper presents an adaptive and compliant approach-to-grasp strategy for multi-finger robotic hands, to improve the performance of autonomous grasping when encountering object position uncertainties. With the proposed approach-to-grasp strategy, the first robot finger to experience unexpected impact would pause its movement in a compliant manner, and remains in contact with the object to minimize the unplanned motion of the target object. At the same time, the remainder of the fingers continuously, adaptively move toward re-adjusted grasping positions with respect to the first finger in contact with the object, without the need for on-line re-planning or re-grasping. An adaptive grasp control strategy based on spatial virtual spring framework is proposed to achieve local (e.g. not resorting to the robotic arm) in-hand adjustments of the fingers not yet in contact. As such, these fingers can be adaptively driven to the adjusted desired position to accomplish the grasp. Experimental results demonstrate that significantly larger position errors with respect to the hand workspace can be accommodated with the proposed adaptive compliant grasp control strategy. As much as 391% increase in position error area coverage has been achieved. Finally, beyond the quantitative analysis, additional observations during the extensive experiment trials are discussed qualitatively, to help examine several open issues, and further understand the approach-to-grasp phases of the robot hand tasks.

Traditional approaches in grasping consider separately the planning of contact points and forces. This often leads to optimally simulated grasps that fail in a real execution when the forces applied at the chosen contact points cannot resist expected perturbations like the own object weight. This paper presents a method that combines a grasp planner with the computation of grasp forces, which provides contact points and contact forces that take into account joint and torque limits of the robotic hand for fingertip grasps. The planner starts with a given hand pose relative to the object, and uses the concept of reachable independent contact regions to obtain a set of force closure grasps. Within this set, the grasp that best counteracts a given external wrench is chosen. The evaluation examples illustrate how the capabilities of the robotic hand can be used more effectively by this joint planning approach.

Many household chores and industrial manufacturing tasks require a certain compliant behavior to make deliberate physical contact with the environment. This compliant behavior can be implemented by modern robotic manipulators. However, in order to plan the task execution, a robot requires generic process models of these tasks which can be adapted to different domains and varying environmental conditions. In this work we propose a classification of compliant manipulation tasks meeting these requirements, to derive related actions for automated planning. We also present a classification for the sub-category of wiping tasks, which are most common and of great importance in service robotics.We categorize actions from an object-centric perspective to make them independent of any specific robot kinematics. The aim of the proposed taxonomy is to guide robotic programmers to develop generic actions for any kind of robotic systems in arbitrary domains.

Generalising dexterous grasps to novel objects is an open problem. We show how to learn grasps for high DoF hands that generalise to novel objects, given as little as one demonstrated grasp. During grasp learning two types of probability density are learned that model the demonstrated grasp. The first density type (the contact model) models the relationship of an individual finger part to local surface features at its contact point. The second density type (the hand configuration model) models the whole hand configuration during the approach to grasp. When presented with a new object, many candidate grasps are generated, and a kinematically feasible grasp is selected that maximises the product of these densities. We demonstrate 31 successful grasps on novel objects (an 86% success rate), transferred from 16 training grasps. The method enables: transfer of dexterous grasps within object categories; across object categories; to and from objects where there is no complete model of the object available; and using two different dexterous hands.

Many houseworks such as cleaning the floor or wiping the windows require to manipulate tools over wide areas. It is necessary to move along a path while manipulating a tool with the whole body and applying exactly the right amount of force to successfully accomplish the task. So mastering such a challenge demands detailed knowledge about the involved objects and the underlying process models. Reasoning about an appropriate parameterization of the task is thereby essential. In this paper we propose a combination of object-centered hybrid reasoning and compliant force control to solve complex whole-body mobile manipulation issues. Depending on the objects involved in the task, an appropriate controller is selected and automatically parameterized. The methods are validated in an elaborate experiment on the humanoid robot Rollin' Justin.

Dexterous multifingered hands are the most complex and versatile variants of robotic end effectors. Compared to simpler grippers and underactuated hands, they should be more capable of grasping and, especially, manipulating different objects. This paper explores the relationship between kinematic design and manipulation performance of robotic hands. Some evaluation criteria frequently used by hand designers to verify kinematic configurations are revisited. The results from these criteria are scrutinized and compared with the evaluation of the manipulation workspace and the ranges of motion of inhand manipulation for a set of predefined objects. Simulations and actual manipulation experiments are carried out with different kinematic configurations on a modular dexterous hand. The results show some disconnection between perceived good designs through common evaluation criteria and their actual, realizable manipulation performance. This work finally gives some insight toward a more holistic approach to design hands that better address grasp and manipulation for the intended tasks and applications.

In-hand manipulation with a multifinger hand is defined as changing the object pose from an initial to a final grasp configuration, while maintaining the fingertip contacts on the object surface. Given only the task constraints, represented as a desired motion of the object and an external force to be applied or resisted, the problem can be expressed as finding a good set of contact points on the object and a corresponding hand configuration compatible with the task to be executed. This paper presents a method for solving such problem, taking into account the kinematic structure and torque limits of the hand, the force closure condition (which must be guaranteed during the whole trajectory), and task compatibility. The feasibility of such method is tested in simulation of 2D and 3D examples.

Independent contact regions have been proposed as a way to overcome possible errors in finger positioning for grasping an object. Efficient implementations for their computation have been developed, that even allow their usage in real-time telemanipulation applications. However, the main problems in the computation of contact regions are that they strongly depend on the initial grasp used as an starting point, and that for a given initial grasp there is not a unique set of contact regions. This paper analyzes the optimality of current approaches for ICR computation in 2D, where the optimal regions are still easily computable, and proposes an algorithm to obtain contact regions closer to the optimal ones. The approach is implemented and analyzed for 2D and 3D objects with any number of contact points.

Classic telepresence approaches allow a human to interact with a remote or a virtual reality environment (VR) with force feedback. Coupling with a remote robot can be used to work in dangerous environments without the human being on-site. The coupling with a VR system can be used for training and verification of task sequences or robotic actions.

Dexterous grasping of objects with uncertain pose is a hard unsolved problem in robotics. This paper solves this problem using information gain re-planning. First we show how tactile information, acquired during a failed attempt to grasp an object can be used to refine the estimate of that object's pose. Second, we show how this information can be used to replan new reach to grasp trajectories for successive grasp attempts. Finally we show how reach-to-grasp trajectories can be modified, so that they maximise the expected tactile information gain, while simultaneously delivering the hand to the grasp configuration that is most likely to succeed. Our main novel outcome is thus to enable tactile information gain planning for Dexterous, high degree of freedom (DoFs) manipulators. We achieve this using a combination of information gain planning, hierarchical probabilistic roadmap planning, and belief updating from tactile sensors for objects with non-Gaussian pose uncertainty in 6 dimensions. The method is demonstrated in trials with simulated robots. Sequential replanning is shown to achieve a greater success rate than single grasp attempts, and trajectories that maximise information gain require fewer re-planning iterations than conventional planning methods before a grasp is achieved.

This paper presents the identification of contact formations using force torque information. As force torque measurements do not map uniquely to their corresponding contact formations, three steps are performed: Initially, the wrench space for each contact formation is computed automatically. Then, a contact formation graph is augmented with a similarity index that reflects the similarity of contact formations with respect to their spanned wrench spaces. A particle filter is used to represent the likeliness of a contact formation given a force torque measurement. Finally, this probability distribution is resolved taking the similarity index, the transitions of the contact formation graph and the history of identified contact formations into account. This allows the recognition of the order of demonstrated contact formations by a measured set of forces and torques. The approach is verified by experiments.

This paper presents an approach for computing power grasps for hands with kinematic structure similar to the human hand, which allows the implementation of strategies inspired in human grasping actions. The proposed method first samples the object surface to look for the best spots for creating an opposing grasp with two or three fingers, and then aligns the other fingers to match the local curvature of the object surface. Different grasp strategies are considered, depending on the relative size of the object with respect to the hand, and on the location of potential obstacles in the environment. Several application examples are provided with two different hand models.

We describe a method for learning planning operators for manipulation tasks from hand-written programs to provide a high-level command interface to a robot manipulator that allows tasks to be specified simply as goals. This is made challenging by the fact that a manipulator is a hybrid system-any model of it consists of discrete variables such as “holding cup” and continuous variables such as the poses of objects and position of the robot. The approach relies on three novel techniques: the action learning from annotated code uses simulation to find PDDL action models corresponding to code fragments. To provide the geometric information needed we use supervised learning to produce a mapping from geometric to symbolic state. The mapping can also be used in reverse to produce a geometric state that makes a set of predicates true, thus allowing desired object positions to be generated during planning. Finally, during execution of the plan we use a partially observable Markov decision problem-based planner to repair the initial plan when unforeseen geometric constraints prevent actions from being executed.

This paper introduces a new framework for task space telemanipulation. The TASK space grasping and MANipulation (TaskMan) concept utilizes a library of tasks based on gesture commands, which replaces the conventional mapping required between the human hand and the end effector. Task communication between the human machine interface (HMI) and the robot end effector requires two symbiotic but nonidentical state machines on the master and slave side. The task states on two sides are synchronized via a single channel communication, as opposed to multi-channel joint space or Cartesian mapped information. HAND gesture command for grasping and MANipulation (HandyMan) HMI command algorithm is proposed for the recognition of hand gestures, which incorporates a library of intuitive task gestures to be used by the teleoperator wearing a CyberGlove. The task gestures are used to drive the states of the TaskMan state machines. With the proposed concepts, this work has realized teleoperated grasp and manipulation with a 15-DoF robot hand in task space. Full 6-DoF of object manipulation was achieved with different grasp combinations, and demonstrated higher repeatability, success rate and easier operation compared to conventional joint space teleoperation methods.

The mobile humanoid Rollin'Justin is a versatile experimental platform for research in manipulation tasks. Previously, different state of the art control methods and first autonomous task execution scenarios have been demonstrated. In this video two new applications with challenging task requirements are presented. One is the catching of one or even two flying balls using all of Justin's degrees of freedom. The other is the autonomous preparation of coffee. Both applications need adequate sensors to support local referencing. The required precision in position and timing is realized in software, using the sensor information, taking the varying precision of Justin's kinematic sub-chains into account and handling all timings in sub-millisecond range.

Independent Contact Regions allow a robust finger placement on the object, despite of potential errors in finger position. They are computed without considering the kinematics of the end-effector, and are usually applied to off-line grasp planners. This paper presents an approach to obtain Reachable Independent Contact Regions by including the hand kinematics in the computational loop. The regions are computed in a short time, which allows real-time applications in virtual grasping. Potential applications of the proposed approach include regrasp planning, and dual-hand manipulation of objects.

It contradicts the human's expectations when humanoid robots move awkwardly during manipulation tasks. The unnatural motion may be caused by awkward start or goal configurations or by probabilistic path planning processes that are often used. This paper shows that the choice of an arm's target configuration strongly effects planning time and how human-like a planned path appears. Human-like goal configurations are found using a criterion from ergonomics research. The knowledge which pose of the Tool Center Point (TCP) can be reached in a natural manner is encapsulated in a restricted reachability map for the robot arm.

This paper presents the graspability map, a novel approach to represent for a particular object the positions and orientations that a given mechanical hand can adopt to achieve a force closure precision grasp. The algorithm is based on the intersection between the fingertip workspaces and the object, plus the verification of a necessary condition for force closure grasps. The maps are computed offline and can be used for comparing the grasp capabilities of different mechanical hands with respect to some benchmark objects. The maps have also potential applications in online grasp and manipulation planning.

This paper presents experimental results on the five-finger dexterous robot hand DLR-HIT II, with Cartesian impedance control based on joint torque and nonlinearity compensation for elastic dexterous robot joints. To improve the performance of the impedance controller, system parameter estimations with extended kalman filter and gravity compensation have been investigated on the robot hand. Experimental results show that, for the harmonic drive robot hand with joint toruqe feedback, accurate position tracking and stable torque/force response can be achieved with cartesian and joint impedance controller. In addition, a FPGA-based control architecture with flexible communication is proposed to perform the designed impedance controller.

This paper introduces a study aimed to help quantify the benefits of limited-performance force-feedback user input devices for space telemanipulation with a dexterous robotic arm. A teleoperated robotic hand has been developed for the European Space Agency by the German Aerospace Center (DLR) for a lunar rover prototype. Studies carried out on this telerobotic system investigated several criteria critical to telemanipulation in space: (1) grasping task completion time, (2) grasping task difficulty, (3) grasp quality, and (4) difficulty level for the operator to assess the grasp quality. Several test subjects were allocated to remotely grasp regular and irregular shaped objects, under different combinations of visual- and force-feedback conditions. This work categorized the benefits of visual- and force-feedback in teleoperated grasping through several performance metrics. Furthermore, it has been shown that, with local joint-level impedance control, good grasping performance with rigid hard objects can be achieved, even with limited force-feedback information and low communication bandwidth. On the other hand, a performance ceiling was also found when grasping deformable objects, where the limited force-feedback setup cannot sufficiently reflect the object boundary to the teleoperator.

In autonomous bimanual operation of a robot, parallelized planning and execution of a task is essential. Elements of a task have different functional and spatial relationships. They may depend on each other and have to be executed in a specific order or they may be independent and their order can be determined freely. Consequently, individual actions can be planned and executed in parallel or not. In a proof of concept, this paper shows that the structure of a task and its mapping onto subordinate planners can significantly influence planning speed and task execution. Independent tasks are planned using two parallel path planners. Dependent tasks are planned using one path planner for both arms. Using a simple, yet expandable experimentation scenario, the resulting recommendations for parameterizing path planners are verified on a humanoid robot. For execution on the real robot a violation of the rigid body model used in path planners had to be addressed.

Research on humanoid robots for use in servicing tasks, e.g. fetching and delivery, attracts steadily more interest. With “Rollin' Justin” a mobile robotic system and research platform is presented that allows sophisticated control algorithms and dexterous manipulation. This video gives an overview of the mobile humanoid robotic system “Rollin' Justin” with special emphasis on mechanical design features, control issues and high-level system capabilities such as human robot interaction.

Research on humanoid robots for use in servicing tasks, e.g. fetching and delivery, attracts steadily more interest. With Rollin' Justin a mobile robotic system and research platform is presented that allows the implementation and demonstration of sophisticated control algorithms and dexterous manipulation. Important problems of service robotics such as mobile manipulation and strategies for using the increased workspace and redundancy in manipulation task can be studied in detail. This paper gives an overview of the design considerations for a mobile platform and their realizations to transform the formerly table-mounted humanoid upper body system Justin into Rollin' Justin, a fully self-sustaining mobile research platform.

Walking is a fascinating way of locomotion that is very robust, especially in unstructured terrain. Many researchers devote their time to understanding its underlying principles and to build robots based on their findings. Using the fingers of DLR-Hand II a six-legged actively compliant walking robot is developed. It is intended to be used as testbed for the evaluation of different force- and position-based leg and gait control algorithms for hexapod walking in rough terrain. Following a brief overview of the finger hardware, the use of fingers as legs is analyzed and discussed. The body geometry as well as the systems constituting the robot are described. The compliance control algorithm used is explained and finally some experimental results are presented.

For mobile manipulators envisioned in home environments a kitchen scenario provides a challenging testbed for numerous skills. Diverse manipulation actions are required, e.g. simple pick and place for moving objects and constrained motions for opening doors and drawers. The robot kinematics and link limits however are restrictive. Therefore especially a constrained trajectory will not be executable from arbitrary placements of the mobile manipulator. A two stage approach is presented to position a mobile manipulator to execute constrained linear trajectories as needed for opening drawers. In a first stage, a representation of a robot armpsilas reachable workspace is computed. Pattern recognition techniques are used to find regions in the workspace representation where these trajectories are possible. A set of trajectories results. In the second stage mobile manipulator placements are computed and the corresponding trajectories are checked for collisions. Compared to a brute force search through the workspace, the success rate of finding a mobile manipulator placement can be increased from 2% to 70%.

This video presents a humanoid two-arm system developed as a research platform for studying dexterous two-handed manipulation. The system is based on the modular DLR-Lightweight-Robot-III and the DLR-Hand-II. Two arms and hands are combined with a three degrees-of-freedom movable torso and a visual system to form a complete humanoid upper body. The diversity of the system is demonstrated by showing the mechanical design, several control concepts, the application of rapid prototyping and hardware-in-the-loop (HIL) development as well as two-handed manipulation experiments and the integration of path planning capabilities.

Humans have at some point learned an abstraction of the capabilities of their arms. By just looking at the scene they can decide which places or objects they can easily reach and which are difficult to approach. Possessing a similar abstraction of a robot arm's capabilities in its workspace is important for grasp planners, path planners and task planners. In this paper, we show that robot arm capabilities manifest themselves as directional structures specific to workspace regions. We introduce a representation scheme that enables to visualize and inspect the directional structures. The directional structures are then captured in the form of a map, which we name the capability map. Using this capability map, a manipulator is able to deduce places that are easy to reach. Furthermore, a manipulator can either transport an object to a place where versatile manipulation is possible or a mobile manipulator or humanoid torso can position itself to enable optimal manipulation of an object.

Autonomous service robots have to recognize and interpret their environment to be able to interact with it. This paper focuses on service tasks such as serving a glass of water where a humanoid two-arm-system has to acquire an object from the scene. A task planner should be able to autonomously discern the necessary actions to solve the task. In the process, a path planner can be used to compute motion sequences to execute these actions. To plan trajectories, the path planner requires a pair of configurations, the start and the goal configuration of the robot, to be provided e.g. by a task planner. This paper proposes a method to autonomously find the goal configurations necessary to acquire objects from the scene and thus makes an attempt to bridge the gap between task planning and path planning. The method determines where to grasp an object by analyzing the scene and the influence of obstacles on the intended grasp location. For the case where the goal object can not be grasped due to obstructing obstacles, a solution is proposed

For the evaluation of grasp quality, different measures have been proposed that are based on wrench spaces. Almost all of them have drawbacks that derive from the non-uniformity of the wrench space, composed of force and torque dimensions. Moreover, many of these approaches are computationally expensive. We address the problem of choosing a proper task wrench space to overcome the problems of the non-uniform wrench space and show how to integrate it in a well-known, high precision and extremely fast computable grasp quality measure.

At our institute, two generations of antropomorphic hands have been designed. In quite a few experiments and demonstrations we could show the abilities of our hands and gain a lot of experience in what artificial hands can do, what abilities they need and where their limitations lie. In this paper, we would like to give an overview over the experiments performed with the DLR hands, our hands abilities and the things that need to be done in the near future.

Many methods for generating and analyzing grasps have been developed in the recent years. They gave insight and comprehension of grasping with robot hands but many of them are rather complicated to implement and of high computational complexity. In this paper we study if the basic quality criterion for grasps, the force-closure property, is in principle easy or difficult to reach. We show that it is not necessary to generate optimal grasps, due to a certain quality measure, for real robot grasping tasks where an average quality grasp is acceptable. We present statistical data that confirm our opinion that a randomized grasp generation algorithm is fast and suitable for the planning of robot grasping tasks.

Usually, grasp planning can be split into two phases. In the first phase one tries to find a set of contacts that allow for stable grasping of an object. In the second phase a feasible hand pose that realizes the grasp with a given hand is calculated. While this point is important for a practical grasp planning system, it has either been considered trivial or been solved by crude heuristics in most cases. Here we present an approach for calculating the hand and finger pose for a given grasp. The problem is formulated as a constraint satisfaction problem and then solved using optimization techniques. The method is applied to two different grasp types: the well known precision grasp and the pinch grasp which is preferred by men when grasping small objects.

In order to grasp and manipulate real world objects, grasp planning systems are required, and they have to be very fast in order to be integrated in online planning systems for robots. This paper presents a method to compute a desirable grasp quality measure very fast and accurately. Based on this measure a heuristic approach towards fast planning of precision grasps for arbitrarily shaped 3D objects is described. A number of feasible grasp candidates are generated heuristically. These grasp candidates are qualified using the described grasp quality measure and the best candidate is chosen. The planned grasps are robust with respect to grasp placement. It is shown that only a relatively small number of grasp candidates has to be generated in order to obtain a good, although not optimal, grasp.