In-hand manipulation and grasping are fundamental yet often separately addressed tasks in robotics. For deriving in-hand manipulation policies, reinforcement learning has recently shown great success. However, the derived controllers are not yet useful in real-world scenarios because they often require a human operator to place the objects in suitable initial (grasping) states. Finding stable grasps that also promote the desired in-hand manipulation goal is an open problem. In this work, we propose a method for bridging this gap by leveraging the critic network of a reinforcement learning agent trained for in-hand manipulation to score and select initial grasps. Our experiments show that this method significantly increases the success rate of in-hand manipulation without requiring additional training. We also present an implementation of a full grasp manipulation pipeline on a real-world system, enabling autonomous grasping and reorientation even of un-wieldy objects. Website: aidx-lab. org/manipulation/icra25

We want to enable fine manipulation with a multi-fingered robotic hand by using modern deep reinforcement learning methods. Key for fine manipulation is a spatially resolved tactile sensor. Here, we present a novel model of a tactile skin that can be used together with rigid-body (hence fast) physics simulators. The model considers the softness of the real fingertips such that a contact can spread across multiple taxels of the sensor depending on the contact geometry. We calibrate the model parameters to allow for an accurate simulation of the real-world sensor. For this, we present a self-contained calibration method without external tools or sensors. To demonstrate the validity of our approach, we learn two challenging fine manipulation tasks: Rolling a marble and a bolt between two fingers. We show in simulation experiments that tactile feedback is crucial for precise manipulation and reaching sub-taxel resolution of <1 mm (despite a taxel spacing of 4 mm). Moreover, we demonstrate that all policies successfully transfer from the simulation to the real robotic hand.Website: aidx-lab.org/skin/iros24

Reorienting diverse objects with a multi-fingered hand is a challenging task. Current methods in robotic in-hand manipulation are either object-specific or require permanent supervision of the object state from visual sensors. This is far from human capabilities and from what is needed in real-world applications. In this work, we address this gap by training shape-conditioned agents to reorient diverse objects in hand, relying purely on tactile feedback (via torque and position measurements of the fingers’ joints). To achieve this, we propose a learning framework that exploits shape information in a reinforcement learning policy and a learned state estimator. We find that representing 3D shapes by vectors from a fixed set of basis points to the shape’s surface, transformed by its predicted 3D pose, is especially helpful for learning dexterous in-hand manipulation. In simulation and real-world experiments, we show the reorientation of many objects with high success rates, on par with state-of-the-art results obtained with specialized single-object agents. Moreover, we show generalization to novel objects, achieving success rates of ~90% even for non-convex shapes.Website: https://aidx-lab.org/manipulation/iros24

Existing grasp controllers usually either only support finger-tip grasps or need explicit configuration of the inner forces. We propose a novel grasp controller that supports arbitrary grasp types, including power grasps with multi-contacts, while operating self-contained on before unseen objects. No detailed contact information is needed, but only a rough 3D model, e.g., reconstructed from a single depth image. First, the external wrench being applied to the object is estimated by using the measured torques at the joints. Then, the torques necessary to counteract the estimated wrench while keeping the object at its initial pose are predicted. The torques are commanded via desired joint angles to an underlying joint-level impedance controller. To reach real-time performance, we propose a learning-based approach that is based on a wrench estimator- and a torque predictor neural network. Both networks are trained in a supervised fashion using data generated via the analytical formulation of the controller. In an extensive simulation-based evaluation, we show that our controller is able to keep 83.1% of the tested grasps stable when applying external wrenches with up to 10 N. At the same time, we outperform the two tested baselines by being more efficient and inducing less involuntary object movement. Finally, we show that the controller also works on the real DLR-Hand II, reaching a cycle time of 6 ms. Website: aidx-lab.org/grasping

Dextrous in-hand manipulation with a multi-fingered robotic hand is a challenging task, esp. when performed with the hand oriented upside down, demanding permanent force-closure, and when no external sensors are used. For the task of reorienting an object to a given goal orientation (vs. infinitely spinning it around an axis), the lack of external sensors is an additional fundamental challenge as the state of the object has to be estimated all the time, e.g., to detect when the goal is reached. In this paper, we show that the task of reorienting a cube to any of the 24 possible goal orientations in a π/2-raster using the torque-controlled DLR-Hand II is possible. The task is learned in simulation using a modular deep reinforcement learning architecture: the actual policy has only a small observation time window of 0.5 s but gets the cube state as an explicit input which is estimated via a deep differentiable particle filter trained on data generated by running the policy. In simulation, we reach a success rate of 92% while applying significant domain randomization. Via zero-shot Sim2Real-transfer on the real robotic system, all 24 goal orientations can be reached with a high success rate. (Web: dlr-alr.github.io/dlr-tactile-manipulation)

When grasping objects with a multi-finger hand, it is crucial for the grasp stability to apply the correct torques at each joint so that external forces are countered. Most current systems use simple heuristics instead of modeling the required torque correctly. Instead, we propose a learning-based approach that is able to predict torques for grasps on unknown objects in real-time. The neural network, trained end-to-end using supervised learning, is shown to predict torques that are more efficient, and the objects are held with less involuntary movement compared to all tested heuristic baselines. Specifically, for 90 % of the grasps the translational deviation of the object is below 2.9 mm and the rotational below 3.1°. To generate training data, we formulate the analytical computation of torques as an optimization problem and handle the indeterminacy of multi-contacts using an elastic model. We further show that the network generalizes to predict torques for unknown objects on the real robot system with an inference time of 1.5 ms. Website: dlr-alr.github.io/grasping/

We show that a purely tactile dextrous in-hand manipulation task with continuous regrasping, requiring permanent force closure, can be learned from scratch and executed robustly on a torque-controlled humanoid robotic hand. The task is rotating a cube without dropping it, but in contrast to OpenAI's seminal cube manipulation task [1], the palm faces downwards and no cameras but only the hand's position and torque sensing are used. Although the task seems simple, it combines for the first time all the challenges in execution as well as learning that are important for using in-hand manipulation in real-world applications. We efficiently train in a precisely modeled and identified rigid body simulation with off-policy deep reinforcement learning, significantly sped up by a domain adapted curriculum, leading to a moderate 600 CPU hours of training time. The resulting policy is robustly transferred to the real humanoid DLR Hand-II, e.g., reaching more than 46 full$2\pi$rotations of the cube in a single run and allowing for disturbances like different cube sizes, hand orientation, or pulling a finger.

We study the problem of estimating the pose of an object which is being manipulated by a multi-fingered robotic hand by only using proprioceptive feedback. To address this challenging problem, we propose a novel variant of differentiable particle filters, which combines two key extensions. First, our learned proposal distribution incorporates recent measurements in a way that mitigates weight degeneracy. Second, the particle update works on non-euclidean manifolds like Lie-groups, enabling learning-based pose estimation in 3D on SE(3). We show that the method can represent the rich and often multi-modal distributions over poses that arise in tactile state estimation. The models are trained in simulation, but by using domain randomization, we obtain state estimators that can be employed for pose estimation on a real robotic hand (equipped with joint torque sensors). Moreover, the estimator runs fast, allowing for online usage with update rates of more than 100 Hz on a single CPU core. We quantitatively evaluate our method and benchmark it against other approaches in simulation. We also show qualitative experiments on the real torque-controlled DLR-Hand II.

Planning collision-free motions for robots with many degrees of freedom is challenging in environments with complex obstacle geometries. Recent work introduced the idea of speeding up the planning by encoding prior experience of successful motion plans in a neural network. However, this “neural motion planning” did not scale to complex robots in unseen 3D environments as needed for real-world applications. Here, we introduce “basis point set”, well-known in computer vision, to neural motion planning as a modern compact environment encoding enabling efficient supervised training networks that generalize well over diverse 3D worlds. Combined with a new elaborate training scheme, we reach a planning success rate of 100 %. We use the network to predict an educated initial guess for an optimization-based planner (OMP), which quickly converges to a feasible solution, massively outperforming random multi-starts when tested on previously unseen environments. For the DLR humanoid Agile Justin with 19 DoF and in challenging obstacle environments, optimal paths can be generated in 200 ms using only a single CPU core. We also show a first successful real-world experiment based on a high-resolution world model from an integrated 3D sensor.

We investigate the problem of planning stable grasps for object manipulations using an 18-DOF robotic hand with four fingers. The main challenge here is the high-dimensional search space, and we address this problem using a novel two-stage learning process. In the first stage, we train an autoregressive network called the hand-pose-generator, which learns to generate a distribution of valid 6D poses of the palm for a given volumetric object representation. In the second stage, we employ a network that regresses 12D finger joint configurations and a scalar grasp quality from given object representations and palm poses. To train our networks, we use synthetic training data generated by a novel grasp planning algorithm, which also proceeds stage-wise: first the palm pose, then the finger positions. Here, we devise a Bayesian Optimization scheme for the palm pose and a physics-based grasp pose metric to rate stable grasps. In experiments on the YCB benchmark data set, we show a grasp success rate of over 83%, as well as qualitative results grasping unknown objects on a real robot system.

n-shot learning, i.e., learning a classifier from only few or even one training samples per class, is the ultimate goal in minimizing the cost of sample acquisition. This is esp. important for active sensing tasks like tactile material classification. Achieving high classification accuracy from only few samples is typically possible only when pre-knowledge is used. In n-shot transfer learning, knowledge from pre-training on a large knowledge set with many classes and samples per class has to be transferred to support the training for a given task set with only few samples per new class. In this paper, we show for the first time that deep end-to-end transfer learning is feasible for tactile material classification. Based on the previously presented (TactNet-II) [1], a deep convolutional neural network (CNN) which reaches superhuman tactile classification performance, we adapt state-of-the art deep transfer learning methods. We evaluate the resulting deep n-shot learning methods with a publicly available tactile material data set with 36 materials [1] in a 6-way n-shot learning task with 30 materials in the knowledge set. In 1-shot learning, our deep transfer learning method reaches 75.5% classification accuracy and in 10-shot more than 90%, outperforming classification without knowledge transfer by more than 40%. This results in an up to 15 time reduction in the number of samples needed to reach a desired accuracy level. We also provide insights of the inner workings of the derived deep transfer learning methods.

Attaching a flexible tactile skin to an existing robotic system is relatively easy compared to integrating most other tactile sensor designs. In this paper we show that material classification purely based on the spatio-temporal signal of a flexible tactile skin can be robustly performed in a real world setting. We compare different classification algorithms and feature sets, including features adopted and extended from previous works in tactile material classification and that are based on the signal's Fourier spectrum. Our convolutional deep learning network architecture, which we also present here, is directly fed with the raw 24000 dimensional sensor signal and performs best by a large margin, reaching a classification accuracy of up to 97.3%.

This video presents the recent upgrades of the mobile humanoid Agile Justin, bringing it closer to an ideal platform for research in autonomous manipulation. Significant upgrades have been made in the fields of mechatronics, 3D sensors, tactile skin, massive GPGPU based computing power, and software communication framework. In addition, first algorithms and two experimental scenarios are presented that take advantage of these new capabilities.

This paper addresses the problem of determining the poses of a pair of inertial sensors mounted at the opposite ends of an imperfect kinematic chain. Due to constraints during design, robots may not be equipped with the required sensors to intrinsically recover the kinematic state up to the required precision. To recover the unknown state using a pair of inertial sensors their relative pose to the opposite ends must be known. We propose an approach to calibrate these unknown relationships in a straightforward and methodically sound way while considering any existing inaccuracies in the kinematic chain. To obtain the desired parameters the calibration problem is formulated as a least squares batch-optimization problem. The proposed approach is integrated on DLR's humanoid robot Agile Justin to determine the pair of inertial sensors mounted at the opposite ends of the torso/head chain and further experimentally validated.

For such common tasks as motion planning or object recognition robots need to perceive their environment and create a dense 3D map of it. A recent breakthrough in this area was the KinectFusion algorithm [16], which relies on step by step matching a depth image to the map via ICP to recover the sensor pose and updating the map based on that pose. In so far it ignores techniques developed in the graph-SLAM area such as fusion with odometry, modeling of uncertainty and distributing an observed inconsistency over the map. This paper presents a method to integrate a dense geometric truncated signed distance function (TSDF) representation as KinectFusion uses with a sparse parametric representation as common in graph SLAM. The key idea is to have local TSDF sub-maps attached to reference nodes in the SLAM graph and derive graph-SLAM links via ICP by matching a map to a depth image. By moving these reference nodes according to the graph-SLAM estimate, the overall map can be deformed without touching individual sub-maps so that re-building of sub-maps is only needed in case of significant deformation within a sub-map. Also, further information can be added to the graph as common in graph SLAM. Examples are odometry or the fact that the ground is roughly but not exactly planar. Additionally, the paper proposes a modification of the KinectFusion algorithm to improve handling of long range data by taking the range dependent uncertainty into account.

This paper studies different criteria for selecting configurations for the task of calibrating a robotic system. Given an automatic and self-contained procedure which allows the robot to calibrate itself without the need of external tools, we are interested in how to select the set of configurations that maximize calibration accuracy while minimizing calibration time. We experiment with the active calibration of a multi-sensorial humanoid's upper body and report that determinant-based criteria should be preferred when a greedy selection is used. In addition to criteria comparison, we further propose a new criterion for configuration selection. Its novelty stems from a direct treatment of the robot's end-effector tool variance. This is contrary to previous approaches which target the variance indirectly via calibration parameters. Our proposed objective function is derived as a compact formulation from the mean error of the robot's end-effector tool from which its variance can be computed using traditional criteria known from the theory of optimal experimental design (e.g. A-optimality).

The Kinect sensor and KinectFusion algorithm have revolutionized environment modeling. We bring these advances to optimization-based motion planning by computing the obstacle and self-collision avoidance objective functions and their gradients directly from the KinectFusion model on the GPU without ever transferring any model to the CPU. Based on this, we implement a proof-of-concept motion planner which we validate in an experiment with a 19-DOF humanoid robot using real data from a tabletop work space. The summed-up time from taking the first look at the scene until the planned path avoiding an obstacle on the table is executed is only three seconds.

Without a precise and up-to-date model of its environment a humanoid robot cannot move safely or act usefully. Ideally, the robot should create a dense 3D environment model in real-time, all the time, and respect obstacle information from it in every move it makes as well as obtain the information it needs for fine manipulation with its fingers from the same map. We propose to use a multi-scale truncated signed distance function (TSDF) map consisting of concentric, nested cubes with exponentially decreasing resolution for this purpose. We show how to extend the KinectFusion real-time SLAM algorithm to the multi-scale case as well as how to compute a multi-scale Euclidean distance transform (EDT) thereby establishing the link to optimization-based planning. We overcome the inability of KinectFusion's localization to handle scenes without enough constraining geometry by switching to mapping-with-known-poses based on forward kinematics. The latter is always available and we know when it is precise. The resulting map has the desired properties: It is computed in real-time (7.5 ms per depth frame for a (8 m)3multi-scale TSDF volume), covers the entire laboratory, does not depend on scene properties (geometry, texture, etc.) and is precise enough to facilitate grasp planning for fine manipulation tasks - all in a single map.

Complex manipulation tasks require an accurate interplay of actuation and sensing. This accuracy can only be achieved by calibrating the relevant components beforehand. Typically calibration procedures are time-consuming and often include subsequent calibration steps, involve multiple people and require external tools. In this paper we alleviate these issues by auto-calibrating the different sensors of DLR's humanoid Rollin' Justin in a single, completely automatic and self-contained procedure, i.e. without calibration plate. By observing a single point feature on each wrist while moving the robot's head, the stereo cameras' intrinsic and extrinsic parameters are calibrated together with the arm joint elasticities and joint angle offsets. Additionally, we use the head motion to calibrate an Inertial Measurement Unit (IMU) extrinsically. Parameters are obtained by formulating the calibration problem as a batch-optimization problem that estimates all parameters jointly. A rough initial guess, as is, e.g., available when re-calibrating, is needed for the estimation and to facilitate marker detection. The procedure is validated on real hardware and reduces the effort considerably allowing rapid (5 min movement time), automatic, and accurate calibration by simply “pushing a button”.

The mobile humanoid Rollin'Justin is a versatile experimental platform for research in manipulation tasks. Previously, different state of the art control methods and first autonomous task execution scenarios have been demonstrated. In this video two new applications with challenging task requirements are presented. One is the catching of one or even two flying balls using all of Justin's degrees of freedom. The other is the autonomous preparation of coffee. Both applications need adequate sensors to support local referencing. The required precision in position and timing is realized in software, using the sensor information, taking the varying precision of Justin's kinematic sub-chains into account and handling all timings in sub-millisecond range.

This paper presents a realtime perception system for catching flying balls with DLR's humanoid Rollin' Justin. We use a two-staged bottom up approach in which we first detect balls as circles and feed these measurements into a multiple hypothesis tracker (MHT). The novel circle detection scheme works in realistic scenes without tuning parameters or background assumptions. We extend the classical multi-hypothesis tracking with prior information about the expected trajectories, therefore limiting the number of hypotheses in the first place. Since the robot starts moving while the ball is still tracked, the cameras shake heavily. A 6-DOF inertial measurements unit (IMU) is integrated to compensate this motion. Using ground-truth from a marker based tracking system we evaluate the metrical accuracy of the motion compensation as well as the tracker's prediction accuracy while in motion.

This video presents the complete procedure for the optimal setup of the DLR MiroSurge telerobotic system for minimally invasive surgery. Two key features are implemented. First, optimization algorithms preoperatively determine several setups that are then rated and selected by the surgeon. Second, the intraoperative situation is taken into account. The newly developed VR-Map device together with fast registration and optimization algorithms enable a quick procedure to assure the optimal patient-specific setup of the robotic system.

We present a real-time self collision detection algorithm applicable for industrial and humanoid robots. The algorithm is based on computing the swept volumes of all bodies and checking them pairwise for collisions. The algorithm operates on joint angle intervals. Such, it does not only test a single or N intermediate configurations but assures safety of a whole movement. Key idea of the new swept volume computation is representing volumes as convex hulls extended by a buffer radius, so called sphere swept convex hulls (SSCH). This leads to tight and compact bounding volumes. The operation set available to model the different joints is strictly conservative and allows for a trade-off between accuracy and computation time. During a configurable timespan the algorithm updates a table of pairwise distances and thus can guarantee hard real-time. It is applied on DLR's humanoid Justin in a sports robotic scenario, where also accuracy and computational performance is evaluated (0.4ms, INTEL T2500@2GHz).

A robotic ball-catching system built from a multi-purpose 7-DOF lightweight arm (DLR-LWR-III) and a 12 DOF four-fingered hand (DLR-Hand-II) is presented. Other than in previous work a mechatronically complex dexterous hand is used for grasping the ball and the decision of where, when and how to catch the ball, while obeying joint, speed and work cell limits, is formulated as an unified nonlinear optimization problem with nonlinear constraints. Three different objective functions are implemented, leading to significantly different robot movements. The high computational demands of an online realtime optimization are met by parallel computation on distributed computing resources (a cluster with 32 CPU cores). The system achieves a catch rate of > 80% and is regularly shown as a live demo at our institute.

This video presents a humanoid two-arm system developed as a research platform for studying dexterous two-handed manipulation. The system is based on the modular DLR-Lightweight-Robot-III and the DLR-Hand-II. Two arms and hands are combined with a three degrees-of-freedom movable torso and a visual system to form a complete humanoid upper body. The diversity of the system is demonstrated by showing the mechanical design, several control concepts, the application of rapid prototyping and hardware-in-the-loop (HIL) development as well as two-handed manipulation experiments and the integration of path planning capabilities.

Mechatronic systems are reaching a new level of complexity, both for the single component and for overall systems making necessary a new software concept for the development and usage of such systems. Here we introduce the agile robot development (aRD) concept, which is a flexible, pragmatic and distributed software design to support and simplify the development of complex mechatronic and robotic systems. It gives easy access to scalable computing performance (even in hard realtime) and is motivated by the abstract view on a robotic system as being a decentral net of calculation blocks and communication links. We discuss design considerations and an implementation of this concept and demonstrate its performance with first applications

We present a system for catching a flying ball with a robot arm using off-the-shelf components (PC based system) for visual tracking. The ball is observed by a large baseline stereo camera, comparing each image to a slowly adapting reference image. We track and predict the target position using an extended Kalman filter, also taking into account the air drag. The calibration is achieved by simply performing a few throws and observing their trajectories, as well as moving the robot to some predefined positions.

In our previous work, integral equation formulations for Mesoscopical, mathematical descriptions of dynamics of popula(cid:173) tions of spiking neurons are getting increasingly important for the understanding of large-scale processes in the brain using simula(cid:173) tions. population dynamics have been derived for a special type of spik(cid:173) ing neurons. For Integrate- and- Fire type neurons, these formula(cid:173) tions were only approximately correct. Here, we derive a math(cid:173) ematically compact, exact population dynamics formulation for Integrate- and- Fire type neurons. It can be shown quantitatively in simulations that the numerical correspondence with microscop(cid:173) ically modeled neuronal populations is excellent. 1 Introduction and motivation The goal of the population dynamics approach is to model the time course of the col(cid:173) lective activity of entire populations of functionally and dynamically similar neurons in a compact way, using a higher descriptionallevel than that of single neurons and spikes. The usual observable at the level of neuronal populations is the population(cid:173) averaged instantaneous firing rate A(t), with A(t)6.t being the number of neurons in the population that release a spike in an interval [t, t+6.t). Population dynamics are formulated in such a way, that they match quantitatively the time course of a given A(t), either gained experimentally or by microscopical, detailed simulation. At least three main reasons can be formulated which underline the importance of the population dynamics approach for computational neuroscience. First, it enables the simulation of extensive networks involving a massive number of neurons and connections, which is typically the case when dealing with biologically realistic functional models that go beyond the single neuron level. Second, it increases the analytical understanding of large-scale neuronal dynamics, opening the way towards better control and predictive capabilities when dealing with large networks. Third, it enables a systematic embedding of the numerous neuronal models operating at different descriptional scales into a generalized theoretic framework, explaining the relationships, dependencies and derivations of the respective models. Early efforts on population dynamics approaches date back as early as 1972, to the work of Wilson and Cowan [8] and Knight [4], which laid the basis for all current population-averaged graded-response models (see e.g. [6] for modeling work using these models). More recently, population-based approaches for spiking neurons were developed, mainly by Gerstner [3, 2] and Knight [5]. In our own previous work [1], we have developed a theoretical framework which enables to systematize and sim(cid:173) ulate a wide range of models for population-based dynamics. It was shown that the equations of the framework produce results that agree quantitatively well with detailed simulations using spiking neurons, so that they can be used for realistic simulations involving networks with large numbers of spiking neurons. Neverthe(cid:173) less, for neuronal populations composed of Integrate-and-Fire (I&F) neurons, this framework was only correct in an approximation. In this paper, we derive the exact population dynamics formulation for I&F neurons. This is achieved by reducing the I&F population dynamics to a point process and by taking advantage of the particular properties of I&F neurons. 2 Background: Integrate-and-Fire dynamics 2.1 Differential form We start with the standard Integrate- and- Fire (I&F) model in form of the well(cid:173) known differential equation [7] (1) which describes the dynamics of the membrane potential Vi of a neuron i that is modeled as a single compartment with RC circuit characteristics. The membrane relaxation time is in this case T = RC with R being the membrane resistance and C the membrane capacitance. The resting potential v R est is the stationary potential that is approached in the no-input case. The input arriving from other neurons is described in form of a current ji. In addition to eq. (1), which describes the integrate part of the I&F model, the neuronal dynamics are completed by a nonlinear step. Every time the membrane potential Vi reaches a fixed threshold () from below, Vi is lowered by a fixed amount Ll > 0, and from the new value of the membrane potential integration according to eq. (1) starts again. if Vi(t) = () (from below) . (2) At the same time, it is said that the release of a spike occurred (i.e., the neuron fired), and the time ti = t of this singular event is stored. Here ti indicates the time of the most recent spike. Storing all the last firing times, we gain the sequence of spikes {t{} (spike ordering index j, neuronal index i). 2.2