We propose a new function estimator, called Wilson Score Kernel Density Estimation, that allows to esti-mate a mean probability and the surrounding confidence interval for parameterized processes with binomiallydistributed outcomes. Our estimator combines the advantages of kernel smoothing, from Kernel Density Esti-mation, and robustness to low number of samples, from Wilson Score. This allows for more robust and dataefficient estimates compared to the individual use of these two estimators. While our estimator is generallyapplicable for processes with binomially distributed outcomes, we will present it in the context of iterativeoptimization. Here we first show the advantage of our estimator on a mathematically well defined problem,and then apply our estimator to an industrial automation process.

A growing trend in computer vision is the use of synthetic images for the evaluation of computer vision algorithms such as 3D pose estimation. This is partly due to the availability of high-quality render engines, which provide highly realistic synthetic images. However, the realism of the rendered images, and thus the reliability of the evaluations, strongly depends on how accurately the scenes are modeled and it requires considerable time and knowledge to do the modeling manually. Automating the modeling process is therefore crucial for making the rendering of photo-realistic synthetic images accessible to the wider robotics community. We present a method for automatically modeling object and light properties for rigid, opaque plastic objects commonly found in industry. Our method relies on recordings of the environment captured with a consumer $360^{\mathrm{o}}$ camera to model the light, and on analysis-by-synthesis to estimate the optical properties of the objects. We show that the synthetic images rendered based on our automatic modeling method can be used to predict the overall performance of a monocular 3D pose estimation algorithm.

Achieving both the flexibility and robustness required to advance the use of robotics in small and medium-sized productions is an essential but difficult task. A fundamental problem is making the robot run blindly without additional sensors while still being robust to uncertainties and variations in the assembly processes. In this paper, we address the use of parameterized motions suitable for blind execution and robust to uncertainties in the assembly process. Collisions and incorrect assemblies are detected based on robot motor currents while motion parameters are updated based on Bayesian Optimization utilizing Gaussian Process learning. This allows for motion parameters to be optimized using real world trials which incorporate all uncertainties inherent in the assembly process without requiring advanced robot and sensor setups. The result is a simple and straightforward system which helps the user automatically find robust and uncertainty-tolerant motions. We present experiments for an assembly case showing both detection and learning in the real world and how these combine to a robust robot system.

Vibratory bowl feeders (VBFs) are a widely used option for industrial part feeding, but their design is still largely manual. A subtask of VBF design is determining an optimal parameter set for the passive devices, called traps, which the VBF uses to ensure correct part orientation. This paper proposes a fast and robust strategy for optimising traps, which makes use of dynamic simulation to efficiently evaluate the performance of parameter sets. The optimisation strategy is based on Bayesian Optimisation and selects new parameter sets to evaluate, using a modified Upper Confidence Bound with regression by Kernel Density Estimation as function estimator. The optimisation is run for four different traps with an industrial part and the best parameter sets are tested for robustness in simulation. The traps are then combined to create two sequences performing orientation of the parts and the designs are prototyped and tested on a real VBF.

In many robotics tasks successful execution requires high precision pose estimates of the objects in the workcell. When the object pose is provided by a computer vision system it is therefore crucial that the vision system is configured such that the required precision is achieved. An important part of the configuration is the sensor placement, however, most work in the field of sensor placement does not take the random, semi-constrained nature of the initial object pose into account. This paper presents a framework which uses an analysis of object stable poses together with dynamic simulation to predict the probability distribution of initial object poses. The framework is highly modular and uses precomputed pose uncertainties and a mixture model to make the integration over all possible stable poses feasible. This makes the framework applicable to a wide range of sensors and uncertainty models. The framework is evaluated in simulation for a concrete example: A single PrimeSense Carmine to be placed at an optimal elevation angle in a table picking scenario where pose uncertainties are modeled using Gaussians.

In robotics, vision sensors are used to estimate the poses of objects in the environment. However, it is a fundamental problem that the estimated poses are not always accurate enough for a given robotic task. Proper sensor placement can mitigate this problem. We present a method which can predict the pose uncertainties in the Iterative Closest Point (ICP) algorithm, which is often used as the last critical pose refinement step in a pose estimation system. With our method we thus provide a crucial tool needed for the optimization of a robust pose estimation system. Our method relies on the generation of synthetic depth images in a Monte Carlo simulation. In this paper we demonstrate our method for depth sensors which rely on Kinect v1 like technology. We evaluate our method using real depth sensor recordings from the publicly available BigBird dataset. The evaluation shows that the uncertainty predictions of our method are in better correspondence with real world experimental results than the state of the art analytical method.

In this paper, we suggest a coherent way of representing results from experiments associated with robotic assembly. The purpose of the representation is to be able to reuse the experiments in other assembly settings. A main novelty in our representation is the inclusion of fine grained experimental uncertainties such as e.g. deviations between a sensed object pose and the actual pose, and we discuss why it is very important for the reusability of experiments to include these uncertainties. Under the reasonable assumption that we can represent the uncertainties as a region around the origin in a potentially high dimensional Cartesian space, we show that we can efficiently represent the studied deviations by storing experiments on a so called spherical lattice. We illustrate that the representation works by studying simulation experiments on two different industrial use cases involving grasping an object and mounting an object on a fixture.

Learning action parameters is becoming an ever more important topic in industrial assembly with tendencies towards smaller batch sizes, more required flexibility and process uncertainties. This paper presents a statistical online learning method capable of handling these issues. The method uses elimination of unpromising parameter sets to reduce the elements of the discretised sample space (inspired by Action Elimination) based on regression uncertainty. Kernel Density Estimation and Wilson Score are explored as internal representations. Based on a dynamic simulator setup for a real world Peg-in-Hole problem, it is shown that the presented method can drastically reduce the number of samples needed. Furthermore, it is also shown that the solution obtained in simulation by our learning method succeeds when executed on the corresponding real world setup.

In this paper, we present an object categorization system capable of assigning multiple and related categories for novel objects using multi-label learning. In this system, objects are described using global geometric relations of 3D features. We propose using the Joint SVM method for learning and we investigate the extraction of hierarchical clusters as a higher-level description of objects to assist the learning. We make comparisons with other multi-label learning approaches as well as single-label approaches (including a state-of-the-art methods using different object descriptors). The experiments are carried out on a dataset of 100 objects belonging to 13 visual and action-related categories. The results indicate that multi-label methods are able to identify the relation between the dependent categories and hence perform categorization accordingly. It is also found that extracting hierarchical clusters does not lead to gain in the system's performance. The results also show that using histograms of global relations to describe objects leads to fast learning in terms of the number of samples required for training.

A current trend in industrial automation implies a need for doing automatic scene understanding, from optical 3D sensors, which in turn imposes a need for a lightweight and reliable 3D optical sensor to be mounted on a collaborative robot e.g., Universal Robot UR5 or Kuka LWR. Here, we empirically evaluate the feasibility of structured light scanners for this purpose, by presenting a system optimized for this task. The system incorporates several recent advances in structured light scanning, such as Large-Gap Gray encoding for dealing with defocusing, automatic creation of illumination masks for noise removal, as well as employing a multi exposure approach dealing with different surface reflectance properties. In addition to this, we investigate expanding the traditional structured light setup to using three cameras, instead of one or two. Also, a novel method for fusing multiple exposures and camera pairs is given. We present an in-depth evaluation, that lead us to conclude, that this setup performs well on tasks relevant for an industrial environment, where many metallic and other surfaces with difficult reflectance properties are in abundance. We demonstrate, that the added components contribute to the robustness of the system. Hereby, we demonstrate that structured light scanning is a technology well suited for hyper flexible industrial automation, by proposing an appropriate system.

Effective robot manipulation requires a vision system which can extract features of the environment which determine what manipulation actions are possible. There is existing work in this direction under the broad banner of recognising “affordances”. We are particularly interested in possibilities for actions afforded by relationships among pairs of objects. For example if an object is “inside” another or “on top” of another. For this there is a need for a vision system which can recognise such relationships in a scene. We use an approach in which a vision system first segments an image, and then considers a pair of objects to determine their physical relationship. The system extracts surface patches for each object in the segmented image, and then compiles various histograms from looking at relationships between the surface patches of one object and those of the other object. From these histograms a classifier is trained to recognise the relationship between a pair of objects. Our results identify the most promising ways to construct histograms in order to permit classification of physical relationships with high accuracy. This work is important for manipulator robots who may be presented with novel scenes and must identify the salient physical relationships in order to plan manipulation activities.

We address the problem of estimating the alignment pose between two models using structure-specific local descriptors. Our descriptors are generated using a combination of 2D image data and 3D contextual shape data, resulting in a set of semi-local descriptors containing rich appearance and shape information for both edge and texture structures. This is achieved by defining feature space relations which describe the neighborhood of a descriptor. By quantitative evaluations, we show that our descriptors provide high discriminative power compared to state of the art approaches. In addition, we show how to utilize this for the estimation of the alignment pose between two point sets. We present experiments both in controlled and real-life scenarios to validate our approach.

In this paper, we present what appears to be the first studies of how to apply learning methods for improving the grasp success probability in industrial bin picking. Our study comprises experiments with both a pneumatic parallel gripper and a suction cup. The baseline is a prioritized list of grasps that have been chosen manually by an experienced engineer. We discuss generally the probability space for success probability in bin picking and we provide suggestions for robust success probability estimates for difference sizes of experimental sets. By performing grasps equivalent to one or two days in production, we show that the success probabilities can be significantly improved by the proposed learning procedure.

We present a method for learning object grasp affordance models in 3D from experience, and demonstrate its applicability through extensive testing and evaluation on a realistic and largely autonomous platform. Grasp affordance refers here to relative object-gripper configurations that yield stable grasps. These affordances are represented probabilistically with grasp densities, which correspond to continuous density functions defined on the space of 6D gripper poses. A grasp density characterizes an object's grasp affordance; densities are linked to visual stimuli through registration with a visual model of the object they characterize. We explore a batch-oriented, experience-based learning paradigm where grasps sampled randomly from a density are performed, and an importance-sampling algorithm learns a refined density from the outcomes of these experiences. The first such learning cycle is bootstrapped with a grasp density formed from visual cues. We show that the robot effectively applies its experience by downweighting poor grasp solutions, which results in increased success rates at subsequent learning cycles. We also present success rates in a practical scenario where a robot needs to repeatedly grasp an object lying in an arbitrary pose, where each pose imposes a specific reaching constraint, and thus forces the robot to make use of the entire grasp density to select the most promising achievable grasp.

Visually extracted 2D and 3D information have their own advantages and disadvantages that complement each other. Therefore, it is important to be able to switch between the different dimensions according to the requirements of the problem and use them together to combine the reliability of 2D information with the richness of 3D information. In this article, we use 2D and 3D information in a feature-based vision system and demonstrate their complementary properties on different applications (namely: depth prediction, scene interpretation, grasping from vision and object learning).