This video presents the recent upgrades of the mobile humanoid Agile Justin, bringing it closer to an ideal platform for research in autonomous manipulation. Significant upgrades have been made in the fields of mechatronics, 3D sensors, tactile skin, massive GPGPU based computing power, and software communication framework. In addition, first algorithms and two experimental scenarios are presented that take advantage of these new capabilities.

This paper addresses the problem of determining the poses of a pair of inertial sensors mounted at the opposite ends of an imperfect kinematic chain. Due to constraints during design, robots may not be equipped with the required sensors to intrinsically recover the kinematic state up to the required precision. To recover the unknown state using a pair of inertial sensors their relative pose to the opposite ends must be known. We propose an approach to calibrate these unknown relationships in a straightforward and methodically sound way while considering any existing inaccuracies in the kinematic chain. To obtain the desired parameters the calibration problem is formulated as a least squares batch-optimization problem. The proposed approach is integrated on DLR's humanoid robot Agile Justin to determine the pair of inertial sensors mounted at the opposite ends of the torso/head chain and further experimentally validated.

We present a method to simultaneously track multiple objects which are subject to physical motion and can be evaluated through raw detector responses in video. Due to their two-staged design, popular tracking-by-detection approaches lack precision in the estimated trajectories due to detector inaccuracies, e.g., lighting, deformation or background clutter. Instead of separating the tasks of detection and tracking, we propose to integrate both in a single probabilistic objective function for determining the object states in a sequence. Both support each other accounting for detection inaccuracies and leading to a robust and precise single target tracker. Based on this, we extend it to multiple targets by solving the problem of determining trajectory limits and sorting out any multiple target ambiguities probabilistically. We apply our method to the task of tracking thrown balls with the goal of accurate trajectory prediction for the purpose of ball catching with a humanoid robot. Our results show improved tracking accuracy with respect to ground truth on average by around 17 %, which is dominated by increased accuracy at the beginning of the trajectory.

This paper studies different criteria for selecting configurations for the task of calibrating a robotic system. Given an automatic and self-contained procedure which allows the robot to calibrate itself without the need of external tools, we are interested in how to select the set of configurations that maximize calibration accuracy while minimizing calibration time. We experiment with the active calibration of a multi-sensorial humanoid's upper body and report that determinant-based criteria should be preferred when a greedy selection is used. In addition to criteria comparison, we further propose a new criterion for configuration selection. Its novelty stems from a direct treatment of the robot's end-effector tool variance. This is contrary to previous approaches which target the variance indirectly via calibration parameters. Our proposed objective function is derived as a compact formulation from the mean error of the robot's end-effector tool from which its variance can be computed using traditional criteria known from the theory of optimal experimental design (e.g. A-optimality).

Complex manipulation tasks require an accurate interplay of actuation and sensing. This accuracy can only be achieved by calibrating the relevant components beforehand. Typically calibration procedures are time-consuming and often include subsequent calibration steps, involve multiple people and require external tools. In this paper we alleviate these issues by auto-calibrating the different sensors of DLR's humanoid Rollin' Justin in a single, completely automatic and self-contained procedure, i.e. without calibration plate. By observing a single point feature on each wrist while moving the robot's head, the stereo cameras' intrinsic and extrinsic parameters are calibrated together with the arm joint elasticities and joint angle offsets. Additionally, we use the head motion to calibrate an Inertial Measurement Unit (IMU) extrinsically. Parameters are obtained by formulating the calibration problem as a batch-optimization problem that estimates all parameters jointly. A rough initial guess, as is, e.g., available when re-calibrating, is needed for the estimation and to facilitate marker detection. The procedure is validated on real hardware and reduces the effort considerably allowing rapid (5 min movement time), automatic, and accurate calibration by simply “pushing a button”.

The mobile humanoid Rollin'Justin is a versatile experimental platform for research in manipulation tasks. Previously, different state of the art control methods and first autonomous task execution scenarios have been demonstrated. In this video two new applications with challenging task requirements are presented. One is the catching of one or even two flying balls using all of Justin's degrees of freedom. The other is the autonomous preparation of coffee. Both applications need adequate sensors to support local referencing. The required precision in position and timing is realized in software, using the sensor information, taking the varying precision of Justin's kinematic sub-chains into account and handling all timings in sub-millisecond range.

This paper presents a realtime perception system for catching flying balls with DLR's humanoid Rollin' Justin. We use a two-staged bottom up approach in which we first detect balls as circles and feed these measurements into a multiple hypothesis tracker (MHT). The novel circle detection scheme works in realistic scenes without tuning parameters or background assumptions. We extend the classical multi-hypothesis tracking with prior information about the expected trajectories, therefore limiting the number of hypotheses in the first place. Since the robot starts moving while the ball is still tracked, the cameras shake heavily. A 6-DOF inertial measurements unit (IMU) is integrated to compensate this motion. Using ground-truth from a marker based tracking system we evaluate the metrical accuracy of the motion compensation as well as the tracker's prediction accuracy while in motion.

This paper shows that the field of visual SLAM has matured enough to build a visual SLAM system from open source components. The system consists of feature detection, data association, and sparse bundle adjustment. For all three modules we evaluate different libraries w.r.t. ground truth.

We describe a method for estimating position and velocity of multiple flying balls for the purpose of robotic ball catching. For this a multi-target recursive Bayes filter, the Gaussian Mixture Probability Hypothesis Density filter (GMPHD), fed by a circle detector is used. This recently developed filter avoids the need to enumerate all possible data association decisions, making them computationally efficient. Over time, a mixture of Gaussians is propagated as tracks, predicted into the future and then sent to the robot. By learning a prior from training data we are focusing on detections that are likely to lead to a catchable trajectory which increases robustness. We evaluate the tracker's performance by comparing it with ground truth data, assessing tracking performance as well as the prediction precision of single tracks. Reasonable prediction performance is acquired right from the start, leading to a good overall catching rate.

Non-linear optimization on constraint graphs has recently been applied very successfully in a variety of SLAM backends. We combine this technique with a principled way of handling non-Euclidean spaces, 3D orientations in particular, based on manifolds to build a generic and very flexible framework, the Manifold Toolkit for Matlab (MTKM). We show that MTKM makes it particularly easy to solve non-trivial multi-sensor calibration problems while remaining generic enough to handle a very different class of problems, namely SLAM, as well: After an introductory example on single camera calibration we apply MTKM to calibration of stereo vision and IMU w.r.t. the kinematic chain of a service robot, RGB-D and accelerometer calibration of a Microsoft Kinect, stereo calibration on a Nao soccer robot, and several SLAM benchmark data sets illustrating MTKM's versatility. MTKM and all presented examples are available as open source from http://openslam.org/MTK.html.