We propose a static, non-overlapping overlay for the Sentinel-2 tiling grid, ensuring that input-output operations do not suffer from the overlap of the original tiles used for storage. For its computation, we use spherical Voronoi polygons, guaranteeing that the distance of the extracted pixels to the centroid of a tile is minimal. In cases where the tiling grid suffers from inconsistencies, e.g., in the area of Norway, local optimization techniques are used to shift or even exclude the points used for polygon computation, i.e., identifying tiles as redundant. Our approach results in fewer data re-projection issues as the extracted pixels are closer to their respective reference systems. As to atmospheric correction, where cloud or terrain shadow corrections usually are less accurate at the edges, it ensures uniform quality as to its static nature independent of seasonal overlapping strategies. Additionally, the static character allows full parallelization of input-output scheduling without the need to use advanced database infrastructures.

In the era of deep learning, training data play an essential role. Yet, their generation is expensive concerning both cost and time. Besides this, it often suffers from (1) insufficient data, (2) fixed definition of categories possibly leading to data imbalance, and (3) difficult quality control of manual annotations. In this paper, we propose an approach for automatic generation of urban building data for detection and classification from remote sensing imagery which attempts to deal with these issues. Datasets in popular format, which can be directly used for training, are created in a fully automatic pipeline from the raw data. Furthermore, the generation process can be customized to adapt the datasets to specific tasks as well as optimized according to the data characteristics of the source. We show that an automatically generated dataset can reach a comparable level to the current open datasets concerning both the quantity of instances as well as the diversity of categories. All this is achieved in a few hours without manual intervention or costs for annotation. Experiments with multiple datasets including the comparison with manually annotated data demonstrate the potential of the proposed approach.

We propose an approach for early crop classification through identifying important timesteps with eXplainable AI (XAI) methods. Our approach consists of training a baseline crop classification model to carry out layer-wise relevance propagation (LRP) so that the salient time step can be identified. We chose a selected number of such important time indices to create the bounding region of the shortest possible classification timeframe. We identified the period 21st April 2019 to 9th August 2019 as having the best trade-off in terms of accuracy and earliness. This timeframe only suffers a 0.75 % loss in accuracy as compared to using the full timeseries. We observed that the LRP-derived important timesteps also highlight small details in input values that differentiates between different classes and possibly offers links to physical crop growth milestones.

The availability of satellite imagery and the surge in popularity of machine learning approaches in remote sensing have created numerous opportunities to study deforestation detection. However, a large amount of labeled data is required for data-driven deep learning methods to achieve acceptable performance. Moreover, labeled datasets are still limited in quantity and quality and can require several years of data acquisition. In this work, we investigate the generalization across sensor modalities of optical satellites for deforestation detection. We argue that exploiting characteristics shared across satellite data, even if acquired by different sensors on board, can significantly reduce the amount of required labeled data. To this end, we explore the use of transfer learning. We observe that a pre-trained neural network outperforms a network trained from scratch.

Training data is an essential ingredient for the development of deep learning approaches. Yet, the preparation of training datasets for building detection and classification in remote sensing images implies substantial manual work and is, therefore, expensive concerning both labor charges and time. Since manual annotation also strongly depends on the experience and expertise of the annotators, quality control is an unavoidable issue. It is, thus, of great interest to explore means to reduce the manual part of dataset generation while keeping the quality of the annotation at an acceptable level.In this paper, we present a novel approach to creating training datasets for individual building detection and classification from remote sensing imagery consisting of a fully automatic pipeline. Using 3D city models and high-resolution imagery as input, annotations including building footprint and their attributes are automatically generated and combined with the corresponding image segments into a standard dataset complying with the COCO format. Experiments comprising also the comparison to manually labeled datasets demonstrate the potential of the proposed work.

With leaps in machine learning techniques and their application on Earth observation challenges has unlocked unprecedented performance across the domain. While the further development of these methods was previously limited by the avail-ability and volume of sensor data and computing resources, the lack of adequate reference data is now constituting new bottlenecks. Since creating such ground-truth information is an expensive and error-prone task, new ways must be devised to source reliable, high-quality reference data on large scales. As an example, we showcase Eurocrops, a reference dataset for crop type classification that aggregates and harmonizes administrative data surveyed in different countries with the goal of transnational interoperability.

Human settlement extent (HSE) and local climate zone (LCZ) maps are both essential sources, e.g., for sustainable urban development and Urban Heat Island (UHI) studies. Remote sensing (RS)- and deep learning (DL)-based classification approaches play a significant role by providing the potential for global mapping. However, most of the efforts only focus on one of the two schemes, usually on a specific scale. This leads to unnecessary redundancies since the learned features could be leveraged for both of these related tasks. In this letter, the concept of multitask learning (MTL) is introduced to HSE regression and LCZ classification for the first time. We propose an MTL framework and develop an end-to-end convolutional neural network (CNN), which consists of a backbone network for shared feature learning, attention modules for task-specific feature learning, and a weighting strategy for balancing the two tasks. We additionally propose to exploit HSE predictions as a prior for LCZ classification to enhance the accuracy. The MTL approach was extensively tested with Sentinel-2 data of 13 cities across the world. The results demonstrate that the framework is able to provide a competitive solution for both tasks.

While in high demand for the development of deep learning approaches, extensive datasets of annotated unmanned aerial vehicle (UAV) imagery are still scarce today. Manual annotation, however, is time-consuming and, thus, has limited the potential for creating large-scale datasets. We tackle this challenge by presenting a procedure for the automatic creation of simulated UAV image sequences in urban areas and pixel-level annotations from publicly available data sources. We synthesize photo-realistic UAV imagery from Google Earth Studio and derive annotations from an open CityGML model that not only provides geometric but also semantic information. The first dataset we exemplarily created using our approach contains 144 000 images of Berlin, Germany, with four types of annotations, namely semantic labels as well as depth, surface normals, and edge maps. Based on this specific case, we demonstrate the entire pipeline and give a comprehensive overview of technical obstacles and solutions. In experiments, we evaluated the quality of our dataset and its potential application in monocular depth estimation and semantic segmentation using deep learning methods. To facilitate the creation of further datasets, we provide our source code along with the produced dataset at https://github.com/sian1995/largescaleuavdataset.

Deep Learning is often criticized as being a black-box method that provides accurate predictions, but a limited explanation of the underlying processes and no indication when to not trust those predictions. Equipping existing deep learning models with an (general) notion of uncertainty can help mitigate both these issues. The Bayesian deep learning community has developed model-agnostic methodology to estimate both data and model uncertainty that can be implemented on top of existing deep learning models. In this work, we test this methodology for deep recurrent satellite time series forecasting and test its assumptions on data and model uncertainty. We tested its effectiveness on an application on climate change where the activity of seasonal vegetation decreased over multiple years.

We describe the workflow of a digital surface models (DSMs) refinement algorithm using a hybrid conditional generative adversarial network (cGAN) where the generative part consists of two parallel networks merged at the last stage forming a WNET architecture. The inputs to the so-called WNET-CGAN are stereo DSMs and panchromatic (PAN) half-meter resolution satellite images. Fusing these helps to propagate fine detailed information from a spectral image and complete the missing 3D knowledge from a stereo DSM about building shapes. Besides, it refines the building outlines and edges making them more rectangular and sharp.

The knowledge about the placement and appearance of lane markings is a prerequisite for the creation of maps with high precision, necessary for autonomous driving, infrastructure monitoring, lanewise traffic management, and urban planning. Lane markings are one of the important components of such maps. Lane markings convey the rules of roads to drivers. While these rules are learned by humans, an autonomous driving vehicle should be taught to learn them to localize itself. Therefore, accurate and reliable lane-marking semantic segmentation in the imagery of roads and highways is needed to achieve such goals. We use airborne imagery that can capture a large area in a short period of time by introducing an aerial lane marking data set. In this paper, we propose a symmetric fully convolutional neural network enhanced by wavelet transform in order to automatically carry out lane-marking segmentation in aerial imagery. Due to a heavily unbalanced problem in terms of a number of lane-marking pixels compared with background pixels, we use a customized loss function as well as a new type of data augmentation step. We achieve a high accuracy in pixelwise localization of lane markings compared with the state-of-the-art methods without using the third-party information. In this paper, we introduce the first high-quality data set used within our experiments, which contains a broad range of situations and classes of lane markings representative of today's transportation systems. This data set will be publicly available, and hence, it can be used as the benchmark data set for future algorithms within this domain.

One of the most unique benefits of multi-baseline synthetic aperture radar interferometry (InSAR) is the long-term monitoring of subtle ground deformation over large areas. Most state-of-the-art algorithms for retrieving such parameter are based on single pixels, e.g. Permanent Scatterer InSAR [1] or clusters of ergodic pixels with stationary phases e.g. SqueeSAR [2]. None of the studies has addressed the joint inversion in an object level, where the true interferometric phase may be varying subject to topography and deformation. Recently, one study has investigated SAR and optical data fusion in order to make use of the rich semantic information from optical images [3]. Based on that work, we seek to investigate the possibility of an object-level multi-baseline InSAR deformation reconstruction given the semantic information from the corresponding optical images. In this paper, we introduced the tensor model for the multi-baseline InSAR inversion and proposed a maximum a posteriori estimator of the deformation parameters by including a spatial prior function in the objective function. Substantial improvement in the deformation estimation is observed in the experiments using both simulated and the real SAR data.

Deformation monitoring by multipass synthetic aperture radar (SAR) interferometry (InSAR) is, so far, the only imaging-based method to assess millimeter-level deformation over large areas from space. Past research mostly focused on the optimal retrieval of deformation parameters on the basis of a single pixel or a pixel cluster. Only until recently, the first demonstration of object-based urban infrastructure monitoring by fusing InSAR and the semantic classification labels derived from optical images was presented by Wanget al.Given such classification labels in the SAR image, we propose a general framework for object-based InSAR parameter retrieval, where the parameters of the whole object are jointly estimated by the inversion of a regularized tensor model instead of pixelwise. Our approach does not assume the stationarity of each sample in the object, which is usually assumed in other pixel cluster-based methods, such as SqueeSAR. In addition, to handle outliers in real data, a robust phase recovery step prior to parameter retrieval is also introduced. In typical settings, the proposed method outperforms the current pixelwise estimators, e.g., periodogram, by a factor of several tens in the accuracy of the linear deformation estimates. Last but not least, for a practical demonstration on bridge monitoring, we present a full workflow of long-term bridge monitoring using the proposed approach.

Deformation monitoring by multi-baseline synthetic aperture radar (SAR) interferometry is so far the only imaging-based method to assess millimeter-level deformation over large areas from space. Past research mostly focused on optimal deformation parameters retrieval on a pixel-basis. Only until recently, the first demonstration of object-based urban infrastructures monitoring by fusing SAR interferometry (InSAR) and the semantic classification labels derived from optical images was presented in [1]–[3]. This paper proposes an algorithm for object-based joint InSAR deformation reconstruction using these classification labels. We derive an object-based multi-baseline InSAR reconstruction model, and propose an efficient algorithm for bridge detection in optical images.

Since depth measuring devices for real-world scenarios became available in the recent past, the use of 3d data now comes more in focus of human action recognition. Due to the increased amount of data it seems to be advisable to model the trajectory of every landmark in the context of all other landmarks which is commonly done by dimensionality reduction techniques like PCA. In this paper we present an approach to directly use the subspaces (i.e. their basis vectors) for extracting features and classification of actions instead of projecting the landmark data themselves. This yields a fixed-length description of action sequences disregarding the number of provided frames. We give a comparison of various global techniques for dimensionality reduction and analyze their suitability for our proposed scheme. Experiments performed on the CMU Motion Capture dataset show promising recognition rates as well as robustness in the presence of noise and incorrect detection of landmarks.