Anomaly Detection is an important problem in industrial processes. Two new subfields have recently emerged: logical anomaly detection and few-shot anomaly detection. The combined task, few-shot logical anomaly detection, has proven exceptionally difficult and highly important for industrial processes. Few-shot methods use suboptimal representations to model composition information necessary for detecting logical anomalies, and previous full-shot methods require a large training set. To solve both problems, we propose ObjectCore, a few-shot logical anomaly detection model that captures the composition information from only a few images without any category-specific information. The composition information of an image is modelled as a collection of object representations. Logical anomalies are detected using bipartite matching between object representations in the test image and object representations in the most similar support image. ObjectCore significantly improves over state-of-the-art methods on two standard benchmarks for few-shot logical anomaly detection, MVTec LOCO and CAD-SD, attaining an image-level AUROC of 80.8% and 96.5%, respectively, in the 4-shot setting. Code: https://github.com/MaticFuc/ObjectCore

• Novel point-based cell detection and classification approach that effectively utilizes multiple proposal candidates. • The largest publicly available manually labeled dataset for point-based cell detection and classification. • An improved evaluation protocol for point-based cell detection and classification. • State-of-the-art results reported on existing public and newly proposed datasets. • First to use pre-trained Transformer-based foundation models for point-based cell detection and classification. We present Patherea, a unified framework for point-based cell detection and classification in histopathology. Our method directly predicts cell locations and classes without intermediate representations and incorporates a hybrid Hungarian matching strategy for more accurate point assignment. Patherea supports flexible backbones and training regimes and leverages pathology foundation models for point-based detection, a capability not feasible with most existing point-based methods. To support method development and benchmarking, we introduce the largest fully manually labeled dataset for Ki-67 IHC, annotated by expert pathologists on full-resolution whole-slide images following standard clinical protocols. We evaluate our approach on existing public point-based datasets—Lizard, BRCA-M2C, and BCData—demonstrating superior F1 scores compared to recent point-based detection methods, while revealing performance saturation on these benchmarks. The Patherea dataset provides a complementary benchmark, capturing clinically relevant low-abundance cell classes where current approaches underperform. Finally, we identify and correct common errors in existing evaluation protocols and provide a benchmarking utility for reproducible assessment. The Patherea dataset and code are publicly released to support future research and fair comparisons.

Abstract Surface-anomaly detection is a critical challenge in ensuring product quality, as defects can pose safety risks and diminish product lifespan. A significant challenge in this domain is the limited availability of anomalous samples which makes training supervised models impractical. In response, unsupervised deep-learning-based methods have attracted significant attention in recent years, as they do not require anomalous samples for training. Such methods assume that during dataset curation all anomalous samples can be identified and subsequently removed from the training set. In practice, however, identifying all anomalous samples without any false negatives is rarely possible, either due to the human errors or due to the ambiguity in what is considered a defect and what is not. In this paper, we address the need to measure the robustness of the unsupervised surface-anomaly detection methods as one of the most important performance metrics. To this end, we propose a robustness measure that describes the sensitivity of an unsupervised method to the presence of anomalous data in the training set. We extensively evaluate seven well established unsupervised methods that follow different anomaly detection paradigms on four diverse datasets and analyze the results. We show that most of the analyzed methods are fairly robust to low percentages of anomalous samples in the training set, with some of them retaining the near-baseline performance even when that percentage grows fairly large.

Recently a growing number of works focus on machine defect detection from anomalous audio patterns. The datasets for the machine audio domain are scarce and recent methods that perform well on benchmarks such as DCASE2020 Task 2, rely on auxiliary information such as annotated data from other training classes in the domain to extract information that can be used in deep-learning classification-based anomaly detection approaches. However, in practical scenarios, annotated data from the same domain may not be readily available so annotation-free methods that can learn appropriate audio representations from unannotated data are needed. We propose AudDSR, a simulation-based anomaly detection method that learns to detect anomalies without additional annotated data and instead focuses on a discrete feature space sampling method for an anomaly simulation process. AudDSR outperforms competing methods that do not rely on annotated data on the DCASE2020 anomalous sound detection benchmark and even matches the performance of some methods that utilize additional annotation information.

RGB-based surface anomaly detection methods have advanced significantly. However, certain surface anomalies remain practically invisible in RGB alone, necessitating the incorporation of 3D information. Existing approaches that employ point-cloud backbones suffer from suboptimal representations and reduced applicability due to slow processing. Re-training RGB backbones, designed for faster dense input processing, on industrial depth datasets is hindered by the limited availability of sufficiently large datasets. We make several contributions to address these challenges. (i) We propose a novel Depth-Aware Discrete Autoencoder (DADA) architecture, that enables learning a general discrete latent space that jointly models RGB and 3D data for 3D surface anomaly detection. (ii) We tackle the lack of diverse industrial depth datasets by introducing a simulation process for learning informative depth features in the depth encoder. (iii) We propose a new surface anomaly detection method 3DSR, which outperforms all existing state-of-theart on the challenging MVTec3D anomaly detection benchmark, both in terms of accuracy and processing speed. The experimental results validate the effectiveness and efficiency of our approach, highlighting the potential of utilizing depth information for improved surface anomaly detection. Code is available at: https://github.com/VitjanZ/3DSR

Robust local navigation is a critical capability for any mobile robot operating in a real-world, unstructured environment, especially when there are humans or other moving obstacles in the workspace. One of the most commonly used methods for local navigation is the Dynamic Window Approach (DWA), which does not address the problem of dynamic obstacles and depends heavily on the settings of the parameters in its cost function. Thus, it is a static approach that does not adapt to the characteristics of the environment, which can change significantly. On the other hand, data-driven deep learning approaches attempt to adapt to the characteristics of the environment by predicting the appropriate robot motion based on the current observation. However, they cannot guarantee collision-free trajectories for unseen inputs. In this work, we combine the best of both worlds. We propose a neural network to predict the weights of the DWA, which is then used for safe local navigation. To address the problem of dynamic obstacles the proposed method considers a short sequence of observations to allow the network to model the motion of the obstacles and adjust the DWA weights accordingly. The network is trained using the Proximal Policy Optimization (PPO) in a reinforcement learning setting in a simulated dynamic environment. We perform a comprehensive evaluation of the proposed approach in realistic scenarios using range scans of real 3D spaces and show that it outperforms both DWA and purely Deep Learning approaches.

Visual surface anomaly detection aims to detect local image regions that significantly deviate from normal appearance. Recent surface anomaly detection methods rely on generative models to accurately reconstruct the normal areas and to fail on anomalies. These methods are trained only on anomaly-free images, and often require hand-crafted post-processing steps to localize the anomalies, which prohibits optimizing the feature extraction for maximal detection capability. In addition to reconstructive approach, we cast surface anomaly detection primarily as a discriminative problem and propose a discriminatively trained reconstruction anomaly embedding model (DRÆM). The proposed method learns a joint representation of an anomalous image and its anomaly-free reconstruction, while simultaneously learning a decision boundary between normal and anomalous examples. The method enables direct anomaly localization without the need for additional complicated post-processing of the network output and can be trained using simple and general anomaly simulations. On the challenging MVTec anomaly detection dataset, DRÆM outperforms the current state-of-the-art unsupervised methods by a large margin and even de-livers detection performance close to the fully-supervised methods on the widely used DAGM surface-defect detection dataset, while substantially outperforming them in localization accuracy. Code at github.com/VitjanZ/DRAEM.

Segmentation-based, two-stage neural network has shown excellent results in the surface defect detection, enabling the network to learn from a relatively small number of samples. In this work, we introduce end-to-end training of the two-stage network together with several extensions to the training process, which reduce the amount of training time and improve the results on the surface defect detection tasks. To enable end-to-end training we carefully balance the contributions of both the segmentation and the classification loss throughout the learning. We adjust the gradient flow from the classification into the segmentation network in order to prevent the unstable features from corrupting the learning. As an additional extension to the learning, we propose frequency-of-use sampling scheme of negative samples to address the issue of over- and under-sampling of images during the training, while we employ the distance transform algorithm on the region-based segmentation masks as weights for positive pixels, giving greater importance to areas with higher probability of presence of defect without requiring a detailed annotation. We demonstrate the performance of the end-to-end training scheme and the proposed extensions on three defect detection datasets-DAGM, KolektorSDD and Severstal Steel defect dataset- where we show state-of-the-art results. On the DAGM and the KolektorSDD we demonstrate 100% detection rate, therefore completely solving the datasets. Additional ablation study performed on all three datasets quantitatively demonstrates the contribution to the overall result improvements for each of the proposed extensions.

Anomaly detection in complex data structures is one of the most challenging problems in computer vision. In many real-world problems, for example in the quality control in modern manufacturing, the anomalous samples are usually rare, resulting in (highly) imbalanced datasets. However, in current research practice, these scenarios are rarely modeled, and as a consequence, evaluation of anomaly detection algorithms often do not reproduce results that are useful for practical applications. First, even in case of highly unbalanced input data, anomaly detection algorithms are expected to significantly reduce the proportion of anomalous samples, detecting “almost all” anomalous samples (with exact specifications depending on the target customer). This places high importance on only the small part of the ROC curve, possibly rendering the standard metrics such as AUC (Area Under Curve) and AP (Average Precision) useless. Second, the target of automatic anomaly detection in practical applications is significant reduction in manual work required, and standard metrics are poor predictor of this feature. Finally, the evaluation may produce erratic results for different randomly initialized training runs of the neural network, producing evaluation results that may not reproduce well in practice. In this paper, we present an evaluation methodology that avoids these pitfalls.

With more research published on adversarial examples, we face a growing need for strong and insightful methods for evaluating the robustness of machine learning solutions against their adversarial threats. Previous work contains problematic and overly simplified evaluation methods, where different methods for generating adversarial examples are compared, even though they produce adversarial examples of differing perturbation magnitudes. This creates a biased evaluation environment, as higher perturbations yield naturally stronger adversarial examples. We propose a novel “accuracy-perturbation curve” that visualizes a classifiers classification accuracy response to adversarial examples of different perturbations. To demonstrate the utility of the curve we perform evaluation of responses of different image classifier architectures to four popular adversarial example methods. We also show how adversarial training improves the robustness of a classifier using the “accuracy-perturbation curve”.

Local navigation is an essential ability of any mobile robot working in a real-world environment. One of the most commonly used methods for local navigation is the Dynamic Window Approach (DWA), which heavily depends on the settings of the parameters in its cost function. Since the optimal choice of the parameters depends on the environment that may significantly vary and change at any time, the parameters should be chosen dynamically in a data-driven way. To cope with this problem, we propose a novel deep convolutional neural network, which dynamically predicts these parameters considering the sensor readings. The network is trained using a state-of-the art reinforcement learning algorithm. In this way, we combine the power of data-driven learning and the dynamic model of the robot, enabling adaptation to the current environment as well as guaranteeing collision-free movement and smooth trajectories of the mobile robot. The experimental results show that the proposed method outperforms the DWA method as well as its recent extension.

Automatic detection and recognition of traffic signs plays a crucial role in management of the traffic-sign inventory. It provides an accurate and timely way to manage traffic-sign inventory with a minimal human effort. In the computer vision community, the recognition and detection of traffic signs are a well-researched problem. A vast majority of existing approaches perform well on traffic signs needed for advanced driver-assistance and autonomous systems. However, this represents a relatively small number of all traffic signs (around 50 categories out of several hundred) and performance on the remaining set of traffic signs, which are required to eliminate the manual labor in traffic-sign inventory management, remains an open question. In this paper, we address the issue of detecting and recognizing a large number of traffic-sign categories suitable for automating traffic-sign inventory management. We adopt a convolutional neural network (CNN) approach, the mask R-CNN, to address the full pipeline of detection and recognition with automatic end-to-end learning. We propose several improvements that are evaluated on the detection of traffic signs and result in an improved overall performance. This approach is applied to detection of 200 traffic-sign categories represented in our novel dataset. The results are reported on highly challenging traffic-sign categories that have not yet been considered in previous works. We provide comprehensive analysis of the deep learning method for the detection of traffic signs with a large intra-category appearance variation and show below 3% error rates with the proposed approach, which is sufficient for deployment in practical applications of the traffic-sign inventory management.

Convolutional neural methods have proven to outperform other approaches in various computer vision tasks. In this paper we apply the deep learning technique to the domain of automated visual surface inspection. We design a unified CNN-based framework for segmentation and detection of surface anomalies. We investigate whether a compact CNN architecture, which exhibit fewer parameters that need to be learned, can be used, while retaining high classification accuracy. We propose and evaluate a compact CNN architecture on a dataset consisting of diverse textured surfaces with variously-shaped weakly-labeled anomalies. The proposed approach achieves state-of-the-art results in terms of anomaly segmentation as well as classification.

The next generation service robots are expected to co-exist with humans in their homes. Such a mobile robot requires an efficient representation of space, which should be compact and expressive, for effective operation in real-world environments. In this paper we present a novel approach for 2D ground-plan-like laser-range-data-based room categorization that builds on a compositional hierarchical representation of space, and show how an additional abstraction layer, whose parts are formed by merging partial views of the environment followed by graph extraction, can achieve improved categorization performance. A new algorithm is presented that finds a dictionary of exemplar elements from a multi-category set, based on the affinity measure defined among pairs of elements. This algorithm is used for part selection in new layer construction. Room categorization experiments have been performed on a challenging publicly available dataset, which has been extended in this work. State-of-the-art results were obtained by achieving the most balanced performance over all categories.

This article presents an integrated robot system capable of interactive learning in dialogue with a human. Such a system needs to have several competencies and must be able to process different types of representations. In this article, we describe a collection of mechanisms that enable integration of heterogeneous competencies in a principled way. Central to our design is the creation of beliefs from visual and linguistic information, and the use of these beliefs for planning system behaviour to satisfy internal drives. The system is able to detect gaps in its knowledge and to plan and execute actions that provide information needed to fill these gaps. We propose a hierarchy of mechanisms which are capable of engaging in different kinds of learning interactions, e.g. those initiated by a tutor or by the system itself. We present the theory these mechanisms are build upon and an instantiation of this theory in the form of an integrated robot system. We demonstrate the operation of the system in the case of learning conceptual models of objects and their visual properties.

Mobile robots need an effective spatial model for the successful operation in real-world environment. The model should be compact and simultaneously possess large expressive power. Moreover, it should scale well. In this paper we propose a new hierarchical representation of space, whose compositional structure is learned based on statistically significant observations. We have focused on a two dimensional space, since many robots perceive their surroundings in two dimensions with the use of a laser range finder or a sonar. We also propose the use of a low-level image descriptor for addressing the room classification problem, by which we demonstrate the performance of our representation. Using only the lower layers of the hierarchy, we obtain state-of-the-art classification results on demanding datasets.

In this paper we present representations and mechanisms that facilitate continuous learning of visual concepts in dialogue with a tutor and show the implemented robot system. We present how beliefs about the world are created by processing visual and linguistic information and show how they are used for planning system behaviour with the aim at satisfying its internal drive - to extend its knowledge. The system facilitates different kinds of learning initiated by the human tutor or by the system itself. We demonstrate these principles in the case of learning about object colours and basic shapes.

For a developmental robotic system to function successfully in the real world, it is important that it be able to form its own internal representations of affordance classes based on observable regularities in sensory data. Usually successful classifiers are built using labeled training data, but it is not always realistic to assume that labels are available in a developmental robotics setting. There does, however, exist an advantage in this setting that can help circumvent the absence of labels: co-occurrence of correlated data across separate sensory modalities over time. The main contribution of this paper is an online classifier training algorithm based on Kohonen's learning vector quantization (LVQ) that, by taking advantage of this co-occurrence information, does not require labels during training, either dynamically generated or otherwise. We evaluate the algorithm in experiments involving a robotic arm that interacts with various household objects on a table surface where camera systems extract features for two separate visual modalities. It is shown to improve its ability to classify the affordances of novel objects over time, coming close to the performance of equivalent fully-supervised algorithms.

We present a general method for integrating visual components into a multi-modal cognitive system. The integration is very generic and can work with an arbitrary set of modalities. We illustrate our integration approach with a specific instantiation of the architecture schema that focuses on integration of vision and language: a cognitive system able to collaborate with a human, learn and display some understanding of its surroundings. As examples of cross-modal interaction we describe mechanisms for clarification and visual learning.

Incremental subspace methods have proven to enable efficient training if large amounts of training data have to be processed or if not all data is available in advance. In this paper we focus on incremental LDA learning which provides good classification results while it assures a compact data representation. In contrast to existing incremental LDA methods we additionally consider reconstructive information when incrementally building the LDA subspace. Hence, we get a more flexible representation that is capable to adapt to new data. Moreover, this allows to add new instances to existing classes as well as to add new classes. The experimental results show that the proposed approach outperforms other incremental LDA methods even approaching classification results obtained by batch learning. 1

Linear subspace methods that provide sufficient reconstruction of the data, such as PCA, offer an efficient way of dealing with missing pixels, outliers, and occlusions that often appear in the visual data. Discriminative methods, such as LDA, which, on the other hand, are better suited for classification tasks, are highly sensitive to corrupted data. We present a theoretical framework for achieving the best of both types of methods: An approach that combines the discrimination power of discriminative methods with the reconstruction property of reconstructive methods which enables one to work on subsets of pixels in images to efficiently detect and reject the outliers. The proposed approach is therefore capable of robust classification with a high-breakdown point. We also show that subspace methods, such as CCA, which are used for solving regression tasks, can be treated in a similar manner. The theoretical results are demonstrated on several computer vision tasks showing that the proposed approach significantly outperforms the standard discriminative methods in the case of missing pixels and images containing occlusions and outliers.

Visual learning is expected to be a continuous and robust process, which treats input images and pixels selectively. In this paper, we present a method for subspace learning, which takes these considerations into account. We present an incremental method, which sequentially updates the principal subspace considering weighted influence of individual images as well as individual pixels within an image. This approach is further extended to enable determination of consistencies in the input data and imputation of the values in inconsistent pixels using the previously acquired knowledge, resulting in a novel incremental, weighted and robust method for subspace learning.

We present an approach to acquisition of range images of objects with non-uniform albedo using a structured light sensor. The main idea is to systematically vary the intensity of the light projector and to form high dynamic scale radiance maps. The range images are then formed from these radiance maps. We tested the method on the objects which have surfaces with very different reflectance properties. We demonstrate that the range images obtained from the high dynamic scale radiance maps are of much better quality, than those obtained directly from the original images of a limited dynamic scale.