We present DESIGNATOR, a toolset for generating datasets for testing and retraining deep neural networks (DNNs) performing computer vision tasks in Martian-like environments. The toolset integrates Marsim, a simulator of the Mars environment, and DESIGNATE, a search-based approach combining simulation, generative adversarial networks (GANs), and search-based test input generation. The tool enables users to select a search strategy, launch simulations in MarsSim, and observe the evolution of simulated images, corresponding realistic images, ground truth labels, model predictions, and fitness values. Beyond supporting researchers and practitioners in generating datasets capable of identifying failures and retraining DNNs, DESIGNATOR can be used as a didactic tool to explain how image datasets can be generated using meta-heuristic search. Furthermore, MarsSim can be used standalone, through its API and GUI. MarsSim enables researchers to assess search-based approaches beyond the predominantly studied automotive context. A demo video of DESIGNATOR is available at https: //youtu.be/fGxgpgViPEo.

In safety-critical systems (e.g., autonomous vehicles and robots), Deep Neural Networks (DNNs) are becoming a key component for computer vision tasks, particularly semantic segmentation. Further, since DNN behavior cannot be assessed through code inspection and analysis, test automation has become an essential activity to gain confidence in the reliability of DNNs. Unfortunately, state-of-the-art automated testing solutions largely rely on simulators, whose fidelity is always imperfect, thus affecting the validity of test results. To address such limitations, we propose to combine meta-heuristic search, used to explore the input space using simulators, with Generative Adversarial Networks (GANs), to transform the data generated by simulators into realistic input images. Such images can be used both to assess the DNN accuracy and to retrain the DNN more effectively. We applied our approach to a state-of-the-art DNN performing semantic segmentation, in two different case studies, and demonstrated that it outperforms a state-of-the-art GAN-based testing solution and several other baselines. Specifically, it leads to the largest number of diverse images leading to the worst DNN accuracy. Further, the images generated with our approach, lead to the highest improvement in DNN accuracy when used for retraining. In conclusion, we suggest to always integrate a trained GAN to transform test inputs when performing search-driven, simulator-based testing.

The adoption of deep neural networks (DNNs) in safety-critical contexts is often prevented by the lack of effective means to explain their results, especially when they are erroneous. In our previous work, we proposed a white-box approach (HUDD) and a black-box approach (SAFE) to automatically characterize DNN failures. They both identify clusters of similar images from a potentially large set of images leading to DNN failures. However, the analysis pipelines for HUDD and SAFE were instantiated in specific ways according to common practices, deferring the analysis of other pipelines to future work. In this article, we report on an empirical evaluation of 99 different pipelines for root cause analysis of DNN failures. They combine transfer learning, autoencoders, heatmaps of neuron relevance, dimensionality reduction techniques, and different clustering algorithms. Our results show that the best pipeline combines transfer learning, DBSCAN, and UMAP. It leads to clusters almost exclusively capturing images of the same failure scenario, thus facilitating root cause analysis. Further, it generates distinct clusters for each root cause of failure, thus enabling engineers to detect all the unsafe scenarios. Interestingly, these results hold even for failure scenarios that are only observed in a small percentage of the failing images.

Deep neural networks (DNNs) have demonstrated superior performance over classical machine learning to support many features in safety-critical systems. Although DNNs are now widely used in such systems (e.g., self driving cars), there is limited progress regarding automated support for functional safety analysis in DNN-based systems. For example, the identification of root causes of errors, to enable both risk analysis and DNN retraining, remains an open problem. In this article, we propose SAFE, a black-box approach to automatically characterize the root causes of DNN errors. SAFE relies on a transfer learning model pre-trained on ImageNet to extract the features from error-inducing images. It then applies a density-based clustering algorithm to detect arbitrary shaped clusters of images modeling plausible causes of error. Last, clusters are used to effectively retrain and improve the DNN. The black-box nature of SAFE is motivated by our objective not to require changes or even access to the DNN internals to facilitate adoption. Experimental results show the superior ability of SAFE in identifying different root causes of DNN errors based on case studies in the automotive domain. It also yields significant improvements in DNN accuracy after retraining, while saving significant execution time and memory when compared to alternatives.

Histopathology images represent a source of assistance for pathologists when diagnosing Cancer. However, in histopathology or in cancer image analysis, pathologists mostly diagnose the pathology as positive if a small part of it is considered cancer tissue. These small parts are called regions of interest (ROI) or patches. Finding the relevant patches is crucial as it can save computation time and memory. Subspace clustering discovers clusters embedded in multiple, overlapping subspaces of high dimensional data. It is an extension of feature selection, which tries to identify relevant subsets of features that are relevant to the clustering process. However, subspace clustering algorithms provide a partition of the data based on one cluster validity measure, assuming a homogeneous similarity measure over the entire data set makes the algorithms not robust to variations in the data characteristics. Therefore, it is beneficial to optimize multiple validity indices simultaneously to capture different aspects of the datasets. The goal of Multi-Objective clustering methods (MOC) is to derive significant clusters by applying two or more objective functions. This paper proposes a new clustering algorithm for patch selection based on subspace and multi-objective clustering to find the data's best partitioning and the images' most relevant patches.