The interaction of robots with their environment requires robust object-centric perception capabilities, typically achieved using learning-based methods trained on synthetic data. However, real-world deployment demands evaluating these capabilities in relevant environments, often involving extensive manual annotation for a quantitative analysis. Additionally, standardized evaluations for robotic tasks, such as grasping, need reproducible object scene configurations and performance benchmarks. We propose a solution to both problems by temporarily employing 3D-printed components, socalled fixtures, which can be designed for any rigid object. Once the scene is set up and object poses are extracted, the fixtures are removed, leaving the natural scene without any artificial distractions. The presented approach is seemingly applicable for pre-determined configurations of multiple objects, which enables precise re-building of scenes with consistent object-toobject relations. Our suggested annotation procedure achieves strong pose accuracy solely on RGB images without any manual involvement. We evaluate and show the usability of the proposed fixtures for automated real-world data annotation to fine-tune a detector and for benchmarking object pose estimation algorithms for robotic grasping. Code and fixture meshes for 3D printing are available at https://github.com/DLRRM/fixture_generation.

Recent research has shown that guiding sampling-based planners with sampling distributions, learned from previous experiences via density estimation, can significantly decrease computation times for motion planning. We propose an algorithm that can estimate the density from the experiences of a robot with different kinematic structure, on the same task. The method allows to generalize collected data from one source manipulator to similarly designed target manipulators, significantly reducing the computation time for new queries for the target manipulator. We evaluate the algorithm in two experiments, including a constrained manipulation task with five different collaborative robots, and show that transferring information can significantly decrease planning time.

One of the main challenges in sampling-based motion planners is to find an efficient sampling strategy. While methods such as Rapidly-exploring Random Tree (RRT) have shown to be more reliable in complex environments than optimization-based methods, they often require longer planning times, which reduces their usability for real-time applications. Recently, biased sampling methods have shown to remedy this issue. For example Gaussian Mixture Models (GMMs) have been used to sample more efficiently in feasible regions of the configuration space. Once the GMM is learned, however, this approach does not adapt its biases to individual planning scene during inference. Hence, we propose in this work a more efficient sampling strategy to further bias the GMM based on visual input upon query. We employ an autoencoder trained entirely in simulation to extract features from depth images and use the latent representation to adjust the weights of each mixture components in the GMM. We show empirically that this improves the sampling efficiency of an RRT motion planner in both real and simulated scenes.

We present repetition sampling, a new adaptive strategy for sampling based planning, which extracts information from previous solutions to focus the search for a similar task on relevant configuration space. We show how to generate distributions for repetition sampling by learning Gaussian Mixture Models from prior solutions. We present how to bias a sampling based planner with the learned distribution to generate new paths for similar tasks. We illustrate our method in a simple maze which explains the generation of the distribution and how repetition sampling can generalize over different environments. We show how to apply repetition sampling to similar constrained manipulation tasks and present our results including significant speedup in execution time when compared to uniform sampling.

We present an incremental method for motion generation in environments with unpredictably moving and initially unknown obstacles. The key to the method is its incremental nature: it locally augments and adapts global motion plans in response to changes in the environment, even if they significantly change the connectivity of the world. The restriction to local changes to a global plan results from the fact that in mobile manipulation, robots can ultimately only rely on their on-board sensors to perceive changes in the world. The proposed method addresses three sub-problems of motion generation with three algorithmic components. The first component reactively adapts plans in response to small, continuous changes. The second augments the plan locally in response to connectivity changes. And the third extracts a global, goal-directed motion from the representation maintained by the first two components. In an experimental evaluation of this method, we show a real-world mobile manipulator executing a whole-body motion task in an initially unknown environment, while incrementally maintaining a plan using only on-board sensors.