Although neural networks have shown impressive results in a multitude of application domains, the "black box" nature of deep learning and lack of confidence estimates have led to scepticism, especially in domains like medicine and physics where such estimates are critical. Research on uncertainty quantification (UQ) has helped elucidate the reliability of these models, but existing implementations of these UQ methods are sparse and difficult to reuse. To this end, we introduce Lightning UQ Box, a PyTorch-based Python library for deep learning-based UQ methods powered by PyTorch Lightning. Lightning UQ Box supports classification, regression, semantic segmentation, and pixelwise regression applications, and UQ methods from a variety of theoretical motivations. With this library, we provide an entry point for practitioners new to UQ, as well as easy-to-use components and tools for scalable deep learning applications.

There is a significant need for principled uncertainty reasoning in machine learning systems as they are increasingly deployed in safety-critical domains. A new approach with uncertainty-aware regression-based neural networks (NNs), based on learning evidential distributions for aleatoric and epistemic uncertainties, shows promise over traditional deterministic methods and typical Bayesian NNs, notably with the capabilities to disentangle aleatoric and epistemic uncertainties. Despite some empirical success of Deep Evidential Regression (DER), there are important gaps in the mathematical foundation that raise the question of why the proposed technique seemingly works. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a heuristic rather than an exact uncertainty quantification. We go on to discuss corrections and redefinitions of how aleatoric and epistemic uncertainties should be extracted from NNs.

Out-of-distribution (OOD) detection is an emerging research topic in remote sensing where existing works focus on single sensor analysis. However, many remote sensing works use multi-modal data to benefit from different characteristics of the sensors. Data that is in-domain for one sensor may be OOD for another sensor. In this work, we address such a scenario focusing on Synthetic Aperture Radar (SAR) and optical data fusion for multi-label scene classification. Besides data distribution shifts caused by unknown classes and snow, we also consider cases where only one modality is affected. Optical images acquired with significant cloud coverage are considered as OOD, while their corresponding SAR images can be in-distribution. We propose a weighted feature propagation strategy based on the in-distribution probabilities of the single modalities. We show, that we not only improve the prediction performance on the cloudy samples but also receive a higher predictive uncertainty when both modalities are OOD.

Deep learning methods have become valuable tools in remote sensing for tasks like aerial scene classification or land cover analysis. Dealing with noisy and very varying data, the need for reliable confidence statements becomes apparent. While deep learning models are known to yield overconfident predictions, quantifying the model uncertainty of those classifiers can help mitigating that effect. Although uncertainty estimation methods for multi-class classification have been published, multi-label classification - the task of labelling data with multiple class labels simultaneously - has hardly been considered yet. In this study, we use multi-label Laplace Approximation to estimate the model uncertainty of deep multi-label classifiers and show how this method can improve calibration and out-of-distribution detection in the remote sensing domain.

Distributional mismatch between training and test data may cause the remote sensing models to behave in unpredictable manner, thus reducing the trustworthiness of such models. Most existing methods for out-of-distribution (OOD) detection rely on availability of OOD samples during training. However, access to OOD data during training is counter intuitive and may be impractical sometimes. Considering this, we propose an unsupervised OOD detection model that does not require training OOD data. The proposed method works by projecting the in-domain samples as a union of 1-dimensional subspaces. Due to the compact feature representation of in-domain samples, OOD samples are less likely to occupy the same feature space, thus they are easily identified. Experimental results demonstrate the capability of the proposed method to detect OOD samples.

In image segmentation, the classic approach of learning a deterministic segmentation neither accounts for noise and ambiguity in the data nor for expert disagreements about the correct segmentation. This has been addressed by architectures that predict heteroscedastic (input-dependent) segmentation uncertainty, which indicates regions of segmentations that should be treated with care. What is missing are structural insights into the uncertainty, which would be desirable for interpretability and systematic adjustments. In the context of state-of-the-art stochastic segmentation networks (SSNs), we solve this issue by dismantling the overall predicted uncertainty into smaller uncertainty components. We obtain them directly from the low-rank Gaussian distribution for the logits in the network head of SSNs, based on a previously unconsidered view of this distribution as a factor model. The rank subsequently encodes a number of latent variables, each of which controls an individual uncertainty component. Hence, we can use the latent variables (called factors) for fine-grained sample control, thereby solving an open problem from previous work. There is one caveat though--factors are only unique up to orthogonal rotations. Factor rotations allow us to structure the uncertainty in a way that endorses simplicity, non-redundancy, and separation among the individual uncertainty components. To make the overall and factor-specific uncertainties at play comprehensible, we introduce flow probabilities that quantify deviations from the mean prediction and can also be used for uncertainty visualization. We show on medical-imaging, earth-observation, and traffic-scene data that rotation criteria based on factor-specific flow probabilities consistently yield the best factors for fine-grained sampling.

Remote sensing deals with a plethora of sensors, a large number of classes/categories, and a huge variation in geography. Due to the difficulty of collecting labeled data uniformly representing all scenarios, data-hungry deep learning models are often trained with labeled data in a source domain that is limited in the above-mentioned aspects. However, during the test/inference phase, such deep learning models are often subjected to a distributional shift, also called out-of-distribution (OOD) samples, in the form of unseen classes, geographic differences, and multisensor differences. Deep learning models can behave in an unexpected manner when subjected to such distributional uncertainties. Vulnerability to OOD data severely reduces the reliability of deep learning models and trusting on such predictions in the absence of any reliability indicator may lead to wrong policy decisions or mishaps in time-bound remote sensing applications. Motivated by this, in this work, we propose a Dirichlet prior network-based model to quantify the distributional uncertainty of deep learning-based remote sensing models. The approach seeks to maximize the representation gap between the in-domain and OOD examples for better segregation of OOD samples at test time. Extensive experiments on several remote sensing image classification datasets demonstrate that the proposed model can quantify distributional uncertainty. To the best of our knowledge, this is the first work to elaborately study distributional uncertainty in context of remote sensing. The codes are publicly available athttps://gitlab.lrz.de/ai4eo/Uncertainty/-/tree/main/DPN-RS.

Understanding and representing traffic patterns are key to detecting anomalous trajectories in the transportation domain. However, some trajectories can exhibit heterogeneous maneuvering characteristics despite confining to normal patterns. Thus, we propose a novel graph-based trajectory representation and association scheme for extraction and confederation of traffic movement patterns, such that data patterns and uncertainty can be learned by deep learning (DL) models. This paper proposes the usage of a recurrent neural network (RNN)-based evidential regression model, which can predict trajectory at future timesteps as well as estimate the data and model uncertainties associated, to detect anomalous maritime trajectories, such as unusual vessel maneuvering, using automatic identification system (AIS) data. Furthermore, we utilize evidential deep learning classifiers to detect unusual turns of vessels and the loss of transmitted signal using predicted class probabilities with associated uncertainties. Our experimental results suggest that the graphical representation of traffic patterns improves the ability of the DL models, such as evidential and Monte Carlo dropout, to learn the temporal-spatial correlation of data and associated uncertainties. Using different datasets and experiments, we demonstrate that the estimated prediction uncertainty yields fundamental information for the detection of traffic anomalies in the maritime and, possibly in other domains.

In remote sensing, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data, differences in the geographic area, and multi-sensor differences. Deep learning based models may behave in unexpected manners when subjected to test data that has such distributional shifts from the training data, also called out-of-distribution (OOD) examples. Vulnerability to OOD data severely reduces the reliability of deep learning based models. In this work, we address this issue by proposing a model to quantify distributional uncertainty of deep learning based remote sensing models. In particular, we adopt a Dirichlet Prior Network for remote sensing data. The approach seeks to maximize the representation gap between the in-domain and OOD examples for a better identification of unknown examples at test time. Experimental results on three exemplary test scenarios show that the proposed model can detect OOD images in remote sensing.

Local Climate Zone (LCZ) classification is the most commonly used scheme to analyze how local urban morphology affects the climate of local areas. Classification methods are often based on remote sensing data or on a fusion of several data sources. In this study, the effects of different fusion strategies of optical and synthetic aperture radar (SAR) data on the accuracy of LCZ classifications are investigated. The data processing is implemented with a convolutional neural network (CNN), where until a fusion layer, separate data sources are processed separately on branches. Strategies of splitting the data into branches and the effects of different fusion stages are compared, together with approaches based on sums of independent classifiers. For our setting, the stage of fusion does not seem to have a big influence on the accuracy. The results of this study contribute to a better understanding of cooperative usage of multispectral and SAR data.