In this study, we examine the potential of several self-supervised deep learning models in predicting forest attributes and detecting forest changes using ESA Sentinel-1 and Sentinel-2 images. The performance of the proposed deep learning models is compared to established conventional machine learning approaches. Studied use-cases include mapping of forest disturbance (windthrown forests, snowload damages) using deep change vector analysis, forest height mapping using UNet+ based models, Momentum contrast and regression modeling. Study areas were represented by several boreal forest sites in Finland. Our results indicate that developed methods allow to achieve superior classification and prediction accuracies compared to traditional methodologies and mimimize the amount of necessary in-situ forestry data.

Tropical deforestation is a major environmental problem with severe consequences such as carbon emissions or biodiversity loss. While much research focuses on monitoring and mapping deforestation, less attention is paid to understanding the various reasons and motivations behind it, known as deforestation drivers. Drivers can typically be identified from optical satellite imagery, but it is often necessary to view the deforested site at multiple points in time to determine the driver, making manual annotation of drivers laborious. In this work, we propose a deep learning model that classifies drivers from time series of Sentinel-2 images. The model combines convolutional, LSTM, and attention layers. To train the model, we use a large crowd-sourced dataset spanning across the tropics. We compare its results to other architectures and show that using time series can bring significant improvement in accuracy compared to single images, especially if a suitable architecture is used. Additionally, we analyze the attention scores produced by our model and show that it learns different strategies for different classes.

In this work, we propose an approach to aid in mapping small settlements, which are often misclassified by models trained on a large-scale context (global or regional). We leverage pre-trained land cover models and few-shot learning to enhance the detection of these settlements. The backbone models are trained globally, but their application is localized through a spatial sampling strategy to address the challenge of detecting missed or unlabelled settlements. The proposed sampling strategy is based on the distance around a test patch and allows for the sampling of both backgrounds (non-settlements) points and settlements. Following this strategy results in a balanced dataset for model fine-tuning and ensures that the model is well-adapted to the local context. The idea is that nearby settlements share more similar properties, which is leveraged in our approach. We evaluate these transferred models by measuring the number of previously unmapped settlements detected by the fine-tuned classifier. For this, we manually annotated over two thousand buildings across two regions of Tanzania, previously unmapped in the original urban landcover product. Our results indicate the potential of the sampling approach, particularly when combined with a model pretrained with Momentum Contrast (MoCo). However, we also highlight the limitations in terms of spatial resolution of Sentinel-2 data for the detection of small settlements.

Human settlement extent (HSE) and local climate zone (LCZ) maps are both essential sources, e.g., for sustainable urban development and Urban Heat Island (UHI) studies. Remote sensing (RS)- and deep learning (DL)-based classification approaches play a significant role by providing the potential for global mapping. However, most of the efforts only focus on one of the two schemes, usually on a specific scale. This leads to unnecessary redundancies since the learned features could be leveraged for both of these related tasks. In this letter, the concept of multitask learning (MTL) is introduced to HSE regression and LCZ classification for the first time. We propose an MTL framework and develop an end-to-end convolutional neural network (CNN), which consists of a backbone network for shared feature learning, attention modules for task-specific feature learning, and a weighting strategy for balancing the two tasks. We additionally propose to exploit HSE predictions as a prior for LCZ classification to enhance the accuracy. The MTL approach was extensively tested with Sentinel-2 data of 13 cities across the world. The results demonstrate that the framework is able to provide a competitive solution for both tasks.

The automatic matching of corresponding pixels in SAR and optical remote sensing imagery has been an active field of research for many years. While early approaches were usually based on the measurement of image similarity by signal-based measures or hand-crafted image features, more recent matching techniques make use of deep learning. Since the different approaches proposed in the literature are usually trained and evaluated on specific, individual datasets, i.e. with unique input data and target label criteria, a direct comparison has not yet been possible. With this paper, we intend to close that gap by providing the first comparative evaluation of different state-of-the-art deep learning-based SAR-optical image matching approaches.

There are many extant approaches for removing clouds from satellite imagery, but they all have significant down-sides, particularly in conducting analysis on a short time series. Schmitt et al. innovated an approach within Google Earth Engine that solves many of those issues, but it contains limitations of its own, specifically in dealing with large regions, determining which date range to draw images from, and its propensity to output hazy or overly-exposed results. In this paper we address these limitations of the Schmitt et al. method and demonstrate how we solved them, both improving upon the algorithm and making it more applicable to a wider variety of use cases.

The automatic matching of corresponding regions in remote sensing imagery acquired by synthetic aperture radar (SAR) and optical sensors is a crucial pre-requesite for many data fusion endeavours such as target recognition, image registration, or 3D-reconstruction by stereogrammetry. Driven by the success of deep learning in conventional optical image matching, we have carried out extensive research with regard to deep matching for SAR-optical multi-sensor image pairs in the recent past. In this paper, we summarize the achieved findings, including different concepts based on (pseudo-)siamese convolutional neural network architectures, hard negative mining, alternative formulations of the underlying loss function, and creation of artificial images by generative adversarial networks. Based on data from state-of-the-art remote sensing missions such as TerraSAR-X, Prism, Worldview-2, and Sentinel-1/2, we show what is already possible today, while highlighting challenges to be tackled by future research endeavors.

In this paper we propose a deep generative framework, based on a generative adversarial network (GAN) and an auto encoder (AE), for generating non-corresponding SAR patches to be used in hard negative mining in situations of limited data quantities. We evaluate the effectiveness of this formulation of hard negative mining for reducing the false positive rate (FPR) and improving network determinability in a SAR-optical patch matching application. Our generative network is trained to generate realistic SAR images using an existing SAR-optical matching dataset. These generated images are then used as non-corresponding, hard negative samples for training a SAR-optical matching network. Our results show that we are able to generate realistic SAR images which exhibit many SAR-like features, such as layover and speckle. We further show that by fine tuning the original matching network using these hard negative samples we are able to improve the overall performance of the original SAR-optical matching network.

In this letter, we propose a pseudo-siamese convolutional neural network architecture that enables to solve the task of identifying corresponding patches in very high-resolution optical and synthetic aperture radar (SAR) remote sensing imagery. Using eight convolutional layers each in two parallel network streams, a fully connected layer for the fusion of the features learned in each stream, and a loss function based on binary cross entropy, we achieve a one-hot indication if two patches correspond or not. The network is trained and tested on an automatically generated data set that is based on a deterministic alignment of SAR and optical imagery via previously reconstructed and subsequently coregistered 3-D point clouds. The satellite images, from which the patches comprising our data set are extracted, show a complex urban scene containing many elevated objects (i.e., buildings), thus providing one of the most difficult experimental environments. The achieved results show that the network is able to predict corresponding patches with high accuracy, thus indicating great potential for further development toward a generalized multisensor key-point matching procedure.