We present a novel large-scale dataset for defect detection in a logistics setting. Recent work on industrial anomaly detection has primarily focused on manufacturing scenarios with highly controlled poses and a limited number of object categories. Existing benchmarks like MVTec-AD [6] and VisA [33] have reached saturation, with state-of-the-art methods achieving up to 99.9% AUROC scores. In contrast to manufacturing, anomaly detection in retail logistics faces new challenges, particularly in the diversity and variability of object pose and appearance. Leading anomaly detection methods fall short when applied to this new setting. To bridge this gap, we introduce a new benchmark that overcomes the current limitations of existing datasets. With over 230,000 images (and more than 29,000 defective instances), it is 40 times larger than MVTec-AD and contains more than 48,000 distinct objects. To validate the difficulty of the problem, we conduct an extensive evaluation of multiple state-of-the-art anomaly detection methods, demonstrating that they do not surpass 56.96% AUROC on our dataset. Further qualitative analysis confirms that existing methods struggle to leverage normal samples under heavy pose and appearance variation. With our large-scale dataset, we set a new benchmark and encourage future research towards solving this challenging problem in retail logistics anomaly detection. The dataset is available for download under https://www.kaputt-dataset.com.

To Perform reliably and consistently over sustained periods of time, large-scale automation critically relies on computer simulation. Simulation allows us and supervisory AI to effectively design, validate, and continuously improve complex processes, and helps practitioners to gain insight into the operation and justify future investments. While numerous successful applications of simulation in industry exist, such as circuit simulation, finite element methods, and computeraided design (CAD), state-of-the-art simulators fall short of accurately modeling physical phenomena, such as friction, impact, and deformation.

We describe the winning entry to the Amazon Picking Challenge 2015. From the experience of building this system and competing, we derive several conclusions: (1) We suggest to characterize robotic system building along four key aspects, each of them spanning a spectrum of solutions - modularity vs. integration, generality vs. assumptions, computation vs. embodiment, and planning vs. feedback. (2) To understand which region of each spectrum most adequately addresses which robotic problem, we must explore the full spectrum of possible approaches. (3) For manipulation problems in unstructured environments, certain regions of each spectrum match the problem most adequately, and should be exploited further. This is supported by the fact that our solution deviated from the majority of the other challenge entries along each of the spectra. This is an abridged version of a conference publication.

We present an integrated approach for perception of unknown articulated objects. To robustly perceive objects and understand interactions, our method tightly integrates pose tracking, shape reconstruction, and the estimation of their kinematic structure. The key insight of our method is that these sub-problems complement each other: for example, tracking is greatly facilitated by knowing the shape of the object, whereas the shape and the kinematic structure can be more easily reconstructed if the motion of the object is known. Our combined method leverages these synergies to improve the performance of perception. We analyze the proposed method in average cases and difficult scenarios using a variety of rigid and articulated objects. The results show that our integrated solution achieves better results than solutions for the individual problems. This demonstrates the benefits of approaching robot perception problems in an integrated manner.

The effectiveness of robot interaction depends on the robot's ability to perform task-relevant actions and on the degree to which it is able to predict the outcomes of these actions. In this paper we argue that the two learning problems - learning actions and learning forward models - must be tightly coupled for each of them to be successful. We present an approach that is able to learn a set of continuous action parameters and relational forward models from the robot's own experience. We formalize our approach as simultaneously clustering experiences in a continuous and a relational representation. Our experiments in a simulated manipulation experiment show that this form of coupled subsymbolic and symbolic learning is required for the robot to acquire task-relevant action capabilities.

We present a method for multi-class segmentation from RGB-D data in a realistic warehouse picking setting. The method computes pixel-wise probabilities and combines them to find a coherent object segmentation. It reliably segments objects in cluttered scenarios, even when objects are translucent, reflective, highly deformable, have fuzzy surfaces, or consist of loosely coupled components. The robust performance results from the exploitation of problem structure inherent to the warehouse setting. The proposed method proved its capabilities as part of our winning entry to the 2015 Amazon Picking Challenge. We present a detailed experimental analysis of the contribution of different information sources, compare our method to standard segmentation techniques, and assess possible extensions that further enhance the algorithm's capabilities. We release our software and data sets as open source.

We introduce a method based on the deflectometry principle for the reconstruction of specular objects exhibiting significant size and geometric complexity. A key feature of our approach is the deployment of an Automatic Virtual Environment (CAVE) as pattern generator. To unfold the full power of this experimental setup, an optical encoding scheme is developed which accounts for the distinctive topology of the CAVE. Furthermore, we devise an algorithm for detecting the object of interest in raw deflect metric images. The segmented foreground is used for single-view reconstruction, the background for estimation of the camera pose, necessary for calibrating the sensor system. Experiments suggest a significant gain of coverage in single measurements compared to previous methods.

To successfully manipulate novel objects, robots must first acquire information about the objects' kinematic structure. We present a method for learning relational kinematic background knowledge from exploratory interactions with the world. As the robot gathers experience, this background knowledge enables the acquisition of kinematic world models with increasing efficiency. Learning such background knowledge, however, proves difficult, especially in complex, feature-rich domains. We present a novel, task-sensitive relational rule learner and demonstrate that it is able to learn accurate kinematic background knowledge in domains where other approaches fail. The resulting background knowledge is more compact and generalizes better than that obtained with existing approaches.

We propose a novel method for object search in realistic environments. We formalize object search as a probabilistic inference problem over possible object locations. The method makes two contributions. First, we identify five priors, each capturing structure inherent to the physical world that is relevant to the search problem. Second, we propose a formalization of the object search problem that leverages these priors. Our formalization in form of a probabilistic graphical model is capable of combining the various sources of information into a consistent probability distribution over object locations. The formalization allows us to sharpen the distribution by propagating the knowledge across locations. We employ the reasoning method to select actions of a searching robot in a simulated environment and show that it results in more efficient object search.

In deflectometry, the shape of mirror objects is recovered from distorted images of a calibrated scene. While remarkably high accuracies are achievable, state-of-the-art methods suffer from two distinct weaknesses: First, for mainly constructive reasons, these can only capture a few square centimeters of surface area at once. Second, reconstructions are ambiguous i.e. infinitely many surfaces lead to the same visual impression. We resolve both of these problems by introducing the first multiview specular stereo approach, which jointly evaluates a series of overlapping deflectometric images. Two publicly available benchmarks accompany this paper, enabling us to numerically demonstrate viability and practicability of our approach.

Grammatical agreement is present in many of the world’s languages today and has become an essential feature that guides linguistic processing. When two words in a sentence are said to “agree”, this means that they share certain features such as “gender”, “number”, “person ” or others. The primary hypothesis of this paper is that marking agreement within one linguistic phrase reduces processing effort as phrasal constituents can more easily be recognized. The drive to reduce processing effort introduces the rise of agreement marking in a population of multiple agents by means of an incrementally aligned mapping between the most discriminatory features of a particular linguistic unit and their associative markers. A series of experiments compare feature selection methods for one-to-one agreement mappings, and show how an agreement system can be bootstrapped.