In this study, we examine the potential of several self-supervised deep learning models in predicting forest attributes and detecting forest changes using ESA Sentinel-1 and Sentinel-2 images. The performance of the proposed deep learning models is compared to established conventional machine learning approaches. Studied use-cases include mapping of forest disturbance (windthrown forests, snowload damages) using deep change vector analysis, forest height mapping using UNet+ based models, Momentum contrast and regression modeling. Study areas were represented by several boreal forest sites in Finland. Our results indicate that developed methods allow to achieve superior classification and prediction accuracies compared to traditional methodologies and mimimize the amount of necessary in-situ forestry data.

Knowledge systems in sustainable agriculture see a big gap with end users due to lack of easy-to-use interfaces with existing knowledge. Adding to the problem, decisions coming from black-box models are not understandable for most users. We try to bridge the gap with an integrated chain of explainable ML models to address the most useful applications in the agri-food industry. To make the integrated model available to users and help them draw benefits out of it, we also propose a novel idea of an explainable ML framework for interaction with human users. This human-in-the-loop approach makes ML models more trustworthy. End-users understand the output from ML models and also improve models with feedback. The application interface is also proposed to have features for multilingual communication among users to build communities. Feedback from communities help further refine ML models. The proposed system is named as Reusable Intelligent solution for Cultivation Enhancement (RISCE). In this article, we provide a demonstration of our system with an intrinsically explainable model for crop vigor analysis.

In this work, we propose an approach to aid in mapping small settlements, which are often misclassified by models trained on a large-scale context (global or regional). We leverage pre-trained land cover models and few-shot learning to enhance the detection of these settlements. The backbone models are trained globally, but their application is localized through a spatial sampling strategy to address the challenge of detecting missed or unlabelled settlements. The proposed sampling strategy is based on the distance around a test patch and allows for the sampling of both backgrounds (non-settlements) points and settlements. Following this strategy results in a balanced dataset for model fine-tuning and ensures that the model is well-adapted to the local context. The idea is that nearby settlements share more similar properties, which is leveraged in our approach. We evaluate these transferred models by measuring the number of previously unmapped settlements detected by the fine-tuned classifier. For this, we manually annotated over two thousand buildings across two regions of Tanzania, previously unmapped in the original urban landcover product. Our results indicate the potential of the sampling approach, particularly when combined with a model pretrained with Momentum Contrast (MoCo). However, we also highlight the limitations in terms of spatial resolution of Sentinel-2 data for the detection of small settlements.

This paper describes an active learning tool for the generation of Earth Observation (EO) benchmark datasets. This tool is able to generate training datasets, based on its active learning strategy with a classification accuracy of around 90%. Afterwards, a data cleaning tool is needed, in order to correct noisy data and provide a clean dataset to be stored in the benchmark database, and for subsequent benchmark verification. The data cleaning procedure is supported by unsupervised learning, using clustering algorithms to group similar patterns, and dimension reduction algorithms to embed them in lower dimension with annotated labels. Moreover, interactive visualizations are implemented in most modules to help better manipulate datasets and get better understandings.

In this paper, we describe an innovative content annotation method for high-resolution Synthetic Aperture Radar (SAR) images generating routinely user-defined semantic labels for sequences of small contiguous image patches, while the full surface areas of our images cover hundreds of km in width and length. Based on this method, we are able to generate a sea-ice dataset that is used in projects to validate the developed machine learning methods.

This paper proposes a novel unsupervised learning method to learn discriminative physics-aware features of Synthetic Aperture Radar images with deep neural networks. We conduct a case study of sea-ice classification using Sentinel-1 Dual-polarized SAR data and the corresponding scattering mechanisms derived from H/α Wishart classification. The scattering mechanisms are encoded as a combination of topics for each SAR image as physics attributes, which guide the deep convolutional neural network to learn physics-aware features automatically. A novel objective function is designed to demonstrate how to conduct the physics-guided learning processing. The experiments show the proposed method can learn discriminative features from SAR images without labeled data, which can achieve a comparable classification result with supervised CNN learning.

The classification of large-scale high-resolution synthetic aperture radar (SAR) land cover images acquired by satellites is a challenging task, facing several difficulties such as semantic annotation with expertise, changing data characteristics due to varying imaging parameters or regional target area differences, and complex scattering mechanisms being different from optical imaging. Given a large-scale SAR land cover data set collected from TerraSAR-X images with a hierarchical three-level annotation of 150 categories and comprising more than 100 000 patches, three main challenges in automatically interpreting SAR images of highly imbalanced classes, geographic diversity, and label noise are addressed. In this letter, a deep transfer learning method is proposed based on a similarly annotated optical land cover data set (NWPU-RESISC45). Besides, a top-2 smooth loss function with cost-sensitive parameters was introduced to tackle the label noise and imbalanced classes' problems. The proposed method shows high efficiency in transferring information from a similarly annotated remote sensing data set, a robust performance on highly imbalanced classes, and is alleviating the overfitting problem caused by label noise. What is more, the learned deep model has a good generalization for other SAR-specific tasks, such as MSTAR target recognition with a state-of-the-art classification accuracy of 99.46%.

This paper describes the work done with the Data Mining components of the H2020 CANDELA project, mainly the Data Model component on the CANDELA platform as its back end, and the user interaction component of the local user machine as front end. The Data Mining tool is basically composed of four main submodules: the Data Model Generation for Data Mining (DMG-DM), the database management system (DBMS) sub-module that has already been dockerized and deployed on the CANDELA platform, the image search and semantic annotation (KDD) sub-module and the multi-knowledge and query (QE) sub-module. They all require user inputs, and connect directly to the database on the platform; they can be started as a normal GUI (Graphical User Interface) tool.

This article presents the achievements of the Candela project. This project aims to develop a platform and new algorithms for the handling, analysis and interpretation of earth observation data. The platform is hosted on the CREODIAS cloud ensuring the proximity of data and its processing. To ensure good performances the platform can scale up or down its computing resources. New algorithms based on machine learning methods for change detection and classification have been developed in the project. The results of these new algorithms are transformed into semantic data used to enrich earth observation products and provide new ways of exploitation. Finally, an end-to-end use of the platform is presented with a use case study of the impact of intense meteorological events on vineyards.

Copernicus is the European programme for monitoring the Earth.It consists of a set of systems that collect data from satellites and in-situ sensors, process this data and provide users with reliable and up-to-date information on a range of environmental and security issues.The data and information processed and disseminated puts Copernicus at the forefront of the big data paradigm, giving rise to all relevant challenges, the so-called 5 Vs: volume, velocity, variety, veracity and value.In this short paper, we discuss the challenges of extracting information and knowledge from huge archives of Copernicus data.We propose to achieve this by scale-out distributed deep learning techniques that run on very big clusters offering virtual machines and GPUs.We also discuss the challenges of achieving scalability in the management of the extreme volumes of information and knowledge extracted from Copernicus data.The envisioned scientific and technical work will be carried out in the context of the H2020 project ExtremeEarth which starts in January 2019.

The increased number of free and open satellite images has led to new applications of these data. Among them is the systematic classification of land cover/use types based on patterns of settlements or agriculture recorded by satellite imagers, in particular, the identification and quantification of temporal changes. In this paper, we will present guidelines and practical examples of how to obtain reliable image patch classification results based on data mining techniques for detecting possible changes that can appear within a data set. Here, we will focus on a scenario, namely forest monitoring using Earth observation Synthetic Aperture Radar data acquired by Sentinel-1, and multispectral data acquired by Sentinel-2.

Deep learning algorithms are widely used in remote sensing image scene understanding. Generally, a large-scale annotated dataset is essential to train a deep neural network for classification. In practical terms, however, a large amount of unknown remote sensing images obtained from different sensors need to be understood which may vary from resolution, geolocation and imaging conditions compared with annotated datasets. In this paper, an unsupervised domain adaptation framework based on ResNet-18 is presented to transfer the knowledge of an existing annotated land cover dataset to other remote sensing data, decreasing the discrepancy among images across sensors. The results show a significant improvement in scene understanding of new remote sensing images.

Advanced interpretation of satellite images calls for automated content analysis as well as interactive content search. A typical example of such systems is EOLib, an ESA funded project that already demonstrated the application potential of TerraSAR-X data within a satellite payload ground segment. In this paper, we analyze the validation results of image content classification using a large set of selected TerraSAR-X images. The classification was done with a cascaded learning method. The main advantage of this method is a coarse-to-fine approach for semantically annotating pixel patches with decreasing size. Once a reliable label is found for a given pixel patch, no further subdivision into still smaller patch sizes is necessary. This leads to a considerable reduction of the computational effort during classification of large-size satellite images.

In this paper, we present a quantitative analysis for a rapid mapping scenario that performs a damage assessment of the 2013 floods in Germany. The scenario is created using pre-disaster and post-disaster TerraSAR-X images and an automated annotation system. Our data set is tiled into patches and Gabor filters are used as a primitive feature method applied to each patch separately. An active learning system based on support vector machine is implemented in order to group the features into categories. Once all categories are identified, these are semantically annotated using reference data as ground truth. In our evaluation 7 categories were retrieved with their specific taxonomies defined using our previous hierarchical annotation scheme. We show that the system supports rapid mapping scenarios (e.g., floods, tsunami, earthquake, etc.) and interactive mapping generation. In addition, with the help of this system, quantitative assessment of disasters can be carried out.

In this paper, we propose to identify the number of categories that can be retrieved from a very high resolution SAR data. The evaluation is done on TerraSAR-X high resolution Spotlight data and the retrieved categories are semi-automatically annotated using as feature vector the Gabor filters; as a classifier the Support Vector Machine, and for ranking the suggested images the relevance feedback. The visualization of the tool was enhanced compared with our previous implementation in order to support the users in his/her approach to search the patches of interest in a large repository. Our dataset consist in 43 scenes that cover as much as possible all the regions over the world. A total of 352 categories are identified that contain urban and non-urban categories.

With the advent of very high resolution (VHR) synthetic aperture radar (SAR) images, local content description is becoming a critical issue for indexing. Conventional SAR image analysis techniques, like segmentation and pixel-level classification, are likely to fail as high-level semantic description should be considered for better discrimination. Therefore, we propose to use image-patch-based analysis method for SAR image interpretation. Inspired by ratio edge detector, in this letter, a new feature extraction method represented by the mean ratios in different directions is proposed for VHR SAR image content characterization. Based on the mean ratio, two simple yet powerful and robust features are proposed for SAR image patch indexing. One is the bag-of-word model using not only the basic statistics, i.e., local mean and variance, but also the mean ratios in different directions. The second one is an adaptation of the Weber local descriptor to SAR images by substituting the gradient with the ratio of mean differences in vertical and horizontal directions. To evaluate the proposed features, image patch indexing based on active learning using a SAR image database consisting of high-resolution TerraSAR-X patches is performed. Comparison with the state-of-the-art features, particularly texture features, has shown improved performance for SAR image categorization.

In this paper, we propose to study the dependence of information extraction technique performance on synthetic aperture radar (SAR) imaging parameters and the selected primitive features (PFs). The evaluation is done on TerraSAR-X data, and the interpretation is realized automatically. In the first part of this paper (use case I), the following issues are analyzed: 1) finding the optimal TerraSAR-X products and their limits of variability and 2) retrieving the number of categories/classes that can be extracted from the TerraSAR-X images using the PFs (gray-level co-occurrence matrix, Gabor filters, quadrature mirror filters, and nonlinear short-time Fourier transform). In the second part of this paper (use case II), we investigate the invariance of the products with the orbit direction and incidence angle. On the one hand, the results show that using ascending looking is better than using descending looking with an average accuracy increase of 7%-8%, approximately. On the other hand, the classification accuracy for the incidence angle varies from a lower value of the incidence to an upper value of the incidence angle (depending on the sensor range) with 4%-5%. The test sites are Venice (Italy), Toulouse (France), Berlin (Germany), and Ottawa (Canada) and are covering as much as possible the huge diversity of modes, types, and geometric resolution configuration of the TerraSAR-X. For the evaluation of all these parameters (resolution, features, orbit looking, and incidence angle), the support-vector-machine classifier is considered. To evaluate the accuracy of the classification, the precision/recall metric is calculated. The first contribution of this paper is the evaluation of different PFs (proposed in the literature for different types of images) and adaptation of these for SAR images. These features are compared (based on the accuracy of the classification) for the first time for a multiresolution pyramid specially built for this purpose. During the evaluation, all the classes were annotated, and a semantic meaning was defined for each class. The second main contribution of this paper is the evaluation of the dependence on the patch size, orbit direction, and incidence angle of the TerraSAR-X. This type of evaluation has not been systematically investigated so far. For the evaluation of the optimal patch, two different patch sizes were defined, with the constrained that the size on ground needs to cover a minimum of one object (e.g., 200 × 200 m on ground). This patch size depends also on the parameters of the data such as resolution and pixel spacing. The investigation of orbit looking and incidence angle is very important for indexing large data sets that has a higher variability of these two parameters. These parameters influence the accuracy of the classification (e.g., if the incidence angle is closer to the lower bounds or closer to the upper bound of the satellite sensor range).

The main purpose of this study is to define for Synthetic Aperture Radar (SAR) data the primitive feature parameters, the incidence angle, and the orbit direction which can be used further for indexing and querying in the EO systems. The evaluation is done on the high resolution SAR data and the interpretation is realized automatically. In this paper, we propose to study and asses the behavior of the primitive feature extracted methods for images of the same scene with two look angles covering the min-max range of the sensor and with ascending / descending orbit looking. The tests are done on TerraSAR-X products Stripmap and high resolution Spotlight, specially and radiometrically enhanced covering the area of Berlin (Germany) and Ottawa (Canada). To identify the optimal primitive features, incident angle, and orbit direction the Support Vector Machine and as a measure of the classification accuracy the precision/recall were considered.

TELEIOS is a recent European project that addresses the need for scalable access to petabytes of Earth Observation data and the discovery and exploitation of knowledge that is hidden in them. TELEIOS builds on scientific database technologies (array databases, SciQL, data vaults) and Semantic Web technologies (stRDF and stSPARQL) implemented on top of a state of the art column store database system (MonetDB). We demonstrate a first prototype of the TELEIOS Virtual Earth Observatory (VEO) architecture, using a forest fire monitoring application as example.