PROTECTED natural areas characterized by minimal modern human footprint are often challenging to assess. Machine learning models, particularly explainable methods, offer promise in understanding and mapping the naturalness of these environments through the analysis of satellite imagery. However, current approaches encounter challenges in delivering valid and objective explanations and quantifying the contribution of specific patterns to naturalness. These challenges persist due to the reliance on hand-crafted weights assigned to contributing patterns, which can introduce subjectivity and limit the model’s ability to capture relationships within the data. We propose the Confident Naturalness Explanation (CNE) framework to address these issues, integrating explainable machine learning and uncertainty quantification. This framework introduces a new quantitative metric to describe the confident contribution of patterns to the concept of naturalness. Additionally, it generates segmentation masks that depict the uncertainty levels in each pixel, highlighting areas where the model lacks knowledge. To showcase the framework’s effectiveness, we apply it to a study site in Fennoscandia, utilizing two open-source satellite datasets. In our proposed metric scale, moors and heathlands register high values of 1 and 0.81, respectively, indicating pronounced naturalness. In contrast, water bodies score lower on the scale, with a metric value of 0.18, placing them at the lower end.

Natural protected areas are vital for biodiversity, climate change mitigation, and supporting ecological processes. Despite their significance, comprehensive mapping is hindered by a lack of understanding of their characteristics and a missing land cover class definition. This paper aims to advance the explanation of the designating patterns forming protected and wild areas. To this end, we propose a novel framework that uses activation maximization and a generative adversarial model. With this, we aim to generate satellite images that, in combination with domain knowledge, are capable of offering complete and valid explanations for the spatial and spectral patterns that define the natural authenticity of these regions. Our proposed framework produces more precise attribution maps pinpointing the designating patterns forming the natural authenticity of protected areas. Our approach fosters our understanding of the ecological integrity of the protected natural areas and may contribute to future monitoring and preservation efforts.

Cauliflower is a hand-harvested crop that must fulfill high-quality standards in sales making the timing of harvest important. However, accurately determining harvest-readiness can be challenging due to the cauliflower head being covered by its canopy. While deep learning enables automated harvest-readiness estimation, errors can occur due to field-variability and limited training data. In this paper, we analyze the reliability of a harvest-readiness classifier with interpretable machine learning. By identifying clusters of saliency maps, we derive reliability scores for each classification result using knowledge about the domain and the image properties. For unseen data, the reliability can be used to (i) inform farmers to improve their decision-making and (ii) increase the model prediction accuracy. Using RGB images of single cauliflower plants at different developmental stages from the GrowliFlower dataset [4], we investigate various saliency mapping approaches and find that they result in different quality of reliability scores. With the most suitable interpretation tool, we adjust the classification result and achieve a 15.72% improvement of the overall accuracy to 88.14% and a 15.44% improvement of the average class accuracy to 88.52% for the GrowliFlower dataset.

Climate change is expected to intensify and increase extreme events in the weather cycle. Since this has a significant impact on various sectors of our life, recent works are concerned with identifying and predicting such extreme events from Earth observations. With respect to wildfire danger forecasting, previous deep learning approaches duplicate static variables along the time dimension and neglect the intrinsic differences between static and dynamic variables. Furthermore, most existing multi-branch architectures lose the interconnections between the branches during the feature learning stage. To address these issues, this paper proposes a 2D/3D two-branch convolutional neural network (CNN) with a Location-aware Adaptive Normalization layer (LOAN). Using LOAN as a building block, we can modulate the dynamic features conditional on their geographical locations. Thus, our approach considers feature properties as a unified yet compound 2D/3D model. Besides, we propose using the sinusoidal-based encoding of the day of the year to provide the model with explicit temporal information about the target day within the year. Our experimental results show a better performance of our approach than other baselines on the challenging FireCube dataset. The results show that location-aware adaptive feature normalization is a promising technique to learn the relation between dynamic variables and their geographic locations, which is highly relevant for areas where remote sensing data builds the basis for analysis. The source code is available at https://github.com/HakamShams/LOAN.

Predicting plant development is an important task in precision farming and an essential metric for decision-making by researchers and farmers. In this work, we propose a novel generative modeling technique for plant growth prediction based on conditional generative adversarial networks. We formulate plant growth as an image-to-image translation task and predict the appearance of a plant growth stage as a function of its previous stage. We take into account that plant growth is inherently multi-modal, depending on numerous and highly variable environmental factors, and thus a single input belongs to a distribution of potential outputs. We encode the ambiguity in an interpretable and low-dimensional latent vector space representing the various factors of variation that are influencing plant growth. We use a novel encoder-based data fusion technique and combine information contained in remote sensing imagery of different cropping systems with data containing the factors of variation to adequately model plant growth. This offers several advantages over existing methods: (1) we show that we can model a distribution of potential appearances and simultaneously outperform existing methods in providing more realistic predictions, (2) the complexity of plant growth is more adequately captured, as various factors influencing plant growth can be included, (3) predictions are controllable by being conditioned by an interpretable latent vector representing the factors of variation along with an input image of a previous growth stage.

Ocean eddies, known as the weather of the ocean, represent gyrating water masses that have horizontal scales from 10 km up to at times 500 km. They transport water mass, heat, nutrition, and carbon and have been identified as hot spots of biological activity. In radar altimetry, they affect alongtrack measurements of sea level height and lead to problems in the subsequent generation of sea level maps. Monitoring eddies is therefore of interest among others to marine biologists, oceanographers, and geodesists. In this paper, using occlusion sensitivity maps (OSMs) we investigate different neural network architectures that address the task of automatic detection of ocean eddies, which is challenging due to their spatio-temporal dynamic behavior. Thus we analyze the importance of the spatial context that is needed to infer correct semantics and compare them between the different architectures. For this, we use data from satellite altimetry since it offers sea surface heights precise enough to expose the presence of eddies. For detection, we utilize a transformer neural network called Teddy which can exploit temporal and spatial information in the data. For evaluating our approach, we use gridded data sets for the area of the western part of the southern Atlantic from 2000 to 2011. Our results are evaluated primarily by employing the dice score metric and show that transformers can infer the semantics with similar performance compared to state-of-the-art CNNs but at the same time are less sensitive towards structural changes due to different modeling of the spatial information of the data.

In Remote Sensing (RS) classification, generalization ability is one of the measure that characterizes the success of Machine Learning (ML) models, but is often impeded by the scarse availability of annotated training data. Annotated RS samples are expensive to obtain and can present large disparities when produced by different annotators. In this paper, we utilize Few-Shot Learning (FSL) with meta-learning to address the challenge of generalization using limited amount of training information. The data used in this paper is leveraged from different datasets that have diverse distributions, that means distinct feature spaces. We tested our approach on publicly available RS benchmark datasets to perform few-shot RS image classification using meta-learning. The results of the experiments suggest that our approach is able to generalize well on the unseen data even with limited number of training samples and reasonable training time.

Object detection is a core task for image analysis and inter-pretation and is broadly applied in applications relying on space- and airborne imagery. Like all supervised deep learning methods, training an object detector generally requires a large amount of representative annotated data, which can be hard to acquire in practice. To overcome this challenge, generating synthetic data can be an option to alleviate a lack of real-world annotated data. One key influential factor for the quality of the synthetic data is the background. We show that the detectors' classifier especially depends severely on the background and has a large impact on the detection preci-sion. Using real background is a natural option, however, we show that this naive approach has drawbacks such as a sig-nificant drop in recall. In this paper, we demonstrate that by using style transfer to match the synthetic foreground to the real background, the detector can mitigate these drawbacks and achieve a more balanced result in terms of precision and recall.

In order to enable the development of powerful machine learning methods for remote sensing-based Earth observation tasks, benchmarks are needed to evaluate the methods and compare them to other methods comprehensively. We present ArtifiVe-Potsdam, a freely available dataset that is targeting vehicle detection in aerial imagery. In particular, the benchmark focuses on enriching real datasets with artificial data and quantifying the added value. The dataset aims to stimulate research on the efficient and cost-effective creation and enrichment of datasets for remote sensing since datasets with a limited number of labels are common and the collection of data and labels is time-consuming and expensive.

The ability to predict the future movements of other vehicles is a subconscious and effortless skill for humans and key to safe autonomous driving. Therefore, trajectory prediction for autonomous cars has gained a lot of attention in recent years. It is, however, still a hard task to achieve human-level performance. Interdependencies between vehicle behaviors and the multimodal nature of future intentions in a dynamic and complex driving environment render trajectory prediction a challenging problem. In this work, we propose a new, datadriven approach for predicting the motion of vehicles in a road environment. The model allows for inferring future intentions from the past interaction among vehicles in highway driving scenarios. Using our neighborhood-based data representation, the proposed system jointly exploits correlations in the spatial and temporal domain using convolutional neural networks. Our system considers multiple possible maneuver intentions and their corresponding motion and predicts the trajectory for five seconds into the future. We implemented our approach and evaluated it on two highway datasets taken in different countries and are able to achieve a competitive prediction performance.

With a limited amount of arable land, increasing demand for food induced by growth in population can only be meet with more effective crop production and more resistant plants. Since crop plants are exposed to many different stress factors, it is relevant to investigate those factors as well as their behavior and reactions. One of the most severe stress factors are diseases, resulting in a high loss of cultivated plants. Our main objective is the forecasting of the spread of disease symptons on barley plants using a Cycle-Consistent Generative Adversarial Network. Our contributions are: (1) we provide a daily forecast for one week to advance research for better planning of plant protection measures, and (2) in contrast to most approaches which use only RGB images, we learn a model with hyperspectral images, providing an information-rich result. In our experiments, we analyze healthy barley leaves and leaves which were inoculated by powdery mildew. Images of the leaves were acquired daily with a hyperspectral microscope, from day 3 to day 14 after inoculation. We provide two methods for evaluating the predicted time series with respect to the reference time series.

A regular monitoring of plants is inevitable to ensure an effective production and to reduce yield losses, for example, caused by different diseases. Infected plants show a visual effect shortly after inoculation. These effects can be understood as anomalies, which do not occur in healthy plant stocks. For automation of harvesting or spraying it is important to recognize anomalies to ensure an on-time reaction by the farmer or breeder. However, these anomalies differ largely in their appearance and a representative model is generally too complex to be learned. Our main objective is reconstruction-based anomaly detection by all-convolutional autoencoder (all-CAE), which combines convolutions with the architecture of an autoencoder (AE). To achieve our objective, we use an hourglass all-convolutional encoder-decoder architecture to create a highly compressed representation in the middle layer. Moreover, we compare different types of noise as regularizer. In our experiments, the method is tested on images of grapes acquired in a vineyard. We show that all-CAE are suitable for anomaly detection and that unnatural noise (salt) shows the best results.

Global climate change plays an essential role in our daily life. Mesoscale ocean eddies have a significant impact on global warming, since they affect the ocean dynamics, the energy as well as the mass transports of ocean circulation. From satellite altimetry we can derive high-resolution, global maps containing ocean signals with dominating coherent eddy structures. The aim of this study is the development and evaluation of a deep-learning based approach for the analysis of eddies. In detail, we develop an eddy identification and tracking framework with two different approaches that are mainly based on feature learning with convolutional neural networks. Furthermore, state-of-the-art image processing tools and object tracking methods are used to support the eddy tracking. In contrast to previous methods, our framework is able to learn a representation of the data in which eddies can be detected and tracked in more objective and robust way. We show the detection and tracking results on sea level anomalies (SLA) data from the area of Australia and the East Australia current, and compare our two eddy detection and tracking approaches to identify the most robust and objective method.

We propose sparse representation-based archetypal graphs as input to spectral clustering for anomaly and change detection. The graph consists of vertices defined by data samples and edges which weights are determines by sparse representation. Besides relationships between all data samples, the graph also encodes the relationship to extremal points, so-called archetypes, which leads to an easily interpretable clustering result. We compare our approach to k-means clustering performed on the original feature representation and to k-means clustering performed on the sparse representation activations. Experiments show that our approach is able to deliver accurate and interpretable results for anomaly and change detection.

This paper presents a sparse-representation-based classification approach with a novel dictionary construction procedure. By using the constructed dictionary, sophisticated prior knowledge about the spatial nature of the image can be integrated. The approach is based on the assumption that each image patch can be factorized into characteristic spatial patterns, also called shapelets, and patch-specific spectral information. A set of shapelets is learned in an unsupervised way, and spectral information is embodied by training samples. A combination of shapelets and spectral information is represented in an undercomplete spatial-spectral dictionary for each individual patch, where the elements of the dictionary are linearly combined to a sparse representation of the patch. The patch-based classification is obtained by means of the representation error. Experiments are conducted on three well-known hyperspectral image data sets. They illustrate that our proposed approach shows superior results in comparison to sparse-representation-based classifiers that use only limited spatial information and behaves competitively with or better than state-of-the-art classifiers utilizing spatial information and kernelized sparse-representation-based classifiers.

This paper introduces archetypal dictionaries for a self-taught learning framework for the application of landcover classification. Self-taught learning, an unsupervised representation learning method, is exploited to learn low-dimensional and discriminative higher-level features, which are used as input into a classification algorithm. Experiments are conducted using a multi-spectral Landsat 5 TM image of a study area in the north of Novo Progresso located in South America. Our results confirm that self-taught learning with archetypal dictionaries provide features, which can be used as input into a linear logistic regression classifier. The obtained classification accuracies are comparable to kernel-based classifier using the original features.

This paper suggests an innovative analysis method for derivation of sea surface heights in coastal areas using conventional radar altimetric waveforms. Our analysis consists of a sub-waveform detection and leading edge identification, while using information from spatially and temporally neighboring waveforms. Sub-waveform detection is done via a sparse representation approach and spatial and temporal information is incorporated by utilizing a conditional random field. Our analysis method is combined with a weighted 3-parameter ocean model retracker. Experiments are conducted using Jason-2 Sensor Geophysical Data Records (SGDR) obtained over the Northern Bay of Bengal in region off the coast of Bangladesh.

This paper presents a superpixel-based classifier for landcover mapping of hyperspectral image data. The approach relies on the sparse representation of each pixel by a weighted linear combination of the training data. Spatial information is incorporated by using a coarse patch-based neighborhood around each pixel as well as data-adapted superpixels. The classification is done via a hierarchical conditional random field, which utilizes the sparse-representation output and models spatial and hierarchical structures in the hyperspectral image. The experiments show that the proposed approach results in superior accuracies in comparison to sparse-representation based classifiers that solely use a patch-based neighborhood.

In this paper, we propose an incremental learning strategy for import vector machines (IVM), which is a sparse kernel logistic regression approach. We use the procedure for the concept of self-training for sequential classification of hyperspectral data. The strategy comprises the inclusion of new training samples to increase the classification accuracy and the deletion of noninformative samples to be memory and runtime efficient. Moreover, we update the parameters in the incremental IVM model without retraining from scratch. Therefore, the incremental classifier is able to deal with large data sets. The performance of the IVM in comparison to support vector machines (SVM) is evaluated in terms of accuracy, and experiments are conducted to assess the potential of the probabilistic outputs of the IVM. Experimental results demonstrate that the IVM and SVM perform similar in terms of classification accuracy. However, the number of import vectors is significantly lower when compared to the number of support vectors, and thus, the computation time during classification can be decreased. Moreover, the probabilities provided by IVM are more reliable, when compared to the probabilistic information, derived from an SVM's output. In addition, the proposed self-training strategy can increase the classification accuracy. Overall, the IVM and its incremental version is worthwhile for the classification of hyperspectral data.

The classification of multisensor data sets, consisting of multitemporal SAR data and multispectral is addressed. In the present study, Import Vector Machines (IVM) are applied on two data sets, consisting of (i) Envisat ASAR/ERS-2 SAR data and a Landsat 5 TM scene, and (h) TerraSAR-X data and a RapidEye scene. The performance of IVM for classifying multisensor data is evaluated and the method is compared to Support Vector Machines (SVM) in terms of accuracy and complexity. In general, the experimental results demonstrate that the classification accuracy is improved by the multisensor data set. Moreover, IVM and SVM perform similar in terms of the classification accuracy. However, the number of import vectors is considerably less than the number of support vectors, and thus the computation time of the IVM classification is lower. IVM can directly be applied to the multi-class problems and provide probabilistic outputs. Overall IVM constitutes a feasible method and alternative to SVM.