Although recently several foundation models for satellite remote sensing imagery have been proposed, they fail to address major challenges of operational applications. Indeed, representations that do not take into account the spectral, spatial and temporal dimensions of the data as well as the irregular or unaligned temporal sampling are of little use for most real world applications. As a consequence, we address some existing shortcomings in the design of foundation models for remote sensing data. In particular, we propose an ALIgned Sits Encoder (ALISE), a novel approach that leverages the spatial, spectral, and temporal dimensions of irregular and unaligned Satellite Image Time Series (SITS), while producing aligned latent representations. ALISE provides easy-to-use fixed-size SITS representations which preserve the spatial resolution of the input SITS required for most mapping tasks. Moreover, to learn informative representations of SITS, we investigate the integration of instance discrimination losses within a masked auto-encoding pre-training task, utilizing a multi-view framework. The model is pre-trained on a custom-built Sentinel-2 multi-year SITS unlabeled data-set. The genericity of the provided representations is assessed on three downstream tasks: crop segmentation, land cover segmentation, and an unsupervised crop change detection task. The results suggest that the use of ALISE’s aligned representations is significantly more effective than previous SSL methods for linear probing segmentation tasks. Additionally, the experiments show the interest of using ALISE representations for unsupervised change detection. Lastly, the impact of the pre-training hyperparameters and the proposed method for aligning irregular and unaligned time series are examined in detail. The code, the pre-trained model as well as the data-sets are released at https://src.koda.cnrs.fr/iris.dumeur/alise.

This work studies change detection from satellite image time series (SITS) with a proposed framework that leverages SITS using self-supervised learning. Experimental evaluation conducted on a study area in southwestern France demonstrates the effectiveness of the approach, with varying quantities of labeled training data. The results highlight the potential of self-supervised learning in producing accurate change detection maps for land cover analysis.

In this paper, a new self-supervised strategy for learning meaningful representations of complex optical Satellite Image Time Series (SITS) is presented. The methodology proposed named U-BARN, a Unet-BERT spAtio-temporal Representation eNcoder, exploits irregularly sampled SITS. The designed architecture allows learning rich and discriminative features from unlabelled data, enhancing the synergy between spatio-spectral and temporal dimensions. To train on unlabelled data, a time series reconstruction pretext task inspired by the BERT strategy is proposed. A Sentinel-2 large-scale unlabelled dataset is used to pre-trained U-BARN. To demonstrate its feature learning capability, representations of SITS encoded by U-BARN, are then used to generate semantic segmentation maps. Experimental results, on a labelled PASTIS dataset, corroborate that accuracies obtained by a shallow classifier using representations learned by the pre-trained model are better than results obtained by the raw SITS. Additionally, a fully supervised experiment is conducted on this same labelled PASTIS dataset to evaluate the effectiveness of the proposed U-BARN architecture. The obtained results show that U-BARN architecture reaches performances similar to the spatio-temporal baseline (U-TAE).

Recent Sentinel satellite constellations and deep learning methods offer great possibilities for estimating the states and dynamics of physical parameters on a global scale. Such parameters and their corresponding uncertainties can be retrieved by machine learning methods solving probabilistic inverse problems. Nevertheless, the scarcity of reference data to train supervised methodologies is a well-known constraint for remote sensing applications. To address such limitations, this work presents a new generic physics-guided probabilistic deep learning methodology to invert physical models. The presented methodology proposes a new strategy to combine probabilistic deep learning methods and physical models avoiding simulation-driven machine learning. The inverse problem is addressed through a Bayesian inference framework by proposing a new physically-constrained self-supervised representation learning methodology. To show the interest of the proposed strategy, the methodology is applied to the retrieval of phenological parameters from NDVI time series. As a result, the probability distributions of the intrinsic phenological model parameters are inferred. The feasibility of the method is evaluated on both simulated and real Sentinel-2 data and compared with different standard algorithms. Promising results show satisfactory accuracy predictions and low inference times for real applications.

One key factor to exhaustive vegetation monitoring lies in the dense temporal sampling of the measurements. Areas subject to multiple human interventions, such as grasslands, are particularly concerned. A Recurrent Neural Network multi-sensor regression approach (SenRVM), relying on the systematic acquisitions of Sentinel-1 SAR satellite, has been thereby proposed. It permits to retrieve vegetation indexes, derived from Sentinel- 2 optical imagery, despite significant cloud cover and with high sampling (6 days). The benefit of SenRVM for filling gaps in vegetation time-series describing agricultural practices is assessed. The proposed approach is compared with classical mono-sensor optical strategies. We adopt a synthetic dataset with large gaps. This realistically mimicks challenging conditions in grassland exploitation detection. Results obtained both for exploited and stable parcels satisfactorily demonstrate the relevance of our approach.

The growing number of images acquired by new satellite missions increases the interest of learning low dimensional image representations without human supervision. Variational AutoEncoders (VAE) are one of the most promising strategies marrying graphical models and deep learning. They are able to learn continuous, structured and probabilistic latent spaces encoding the data. In this work, a VAE architecture is proposed and analyzed for multispectral Sentinel-2 images. The regularized β-VAE is studied and compared with the classical auto-encoder strategy. A classification experiment is carried out to corroborate that generated latent spaces can preserve the salient features of the input data. Classification results show that high accuracies can be obtained by using low dimensional latent representation learned by VAE as input data in a standard classification approach.

Cloudy conditions impede and reduce the utility of optical imagery. With the launch of Sentinel-1A and B, the ongoing availability of RADARSAT-2 imagery, and the expected launch of the RADARSAT Constellation Mission (RCM), dense time series of C-band Synthetic Aperture Radar (SAR) data will now be readily available. For crop classification and mapping, SAR imagery has yet to be used to its full potential and has generally been combined with optical imagery. The JECAM SAR Inter-Comparison Experiment is a multi-year, multi-partner project that aims to compare global methods for SAR-based crop monitoring and inventory. Sets of dense time-series SAR imagery which include RADARSAT-2 and Sentinel-1 data were prepared for this experiment. AAFC's operational Decision Tree (DT) and newly implemented Random Forest (RF) classification methodologies were applied to these SAR only data-stacks, and to optimized, traditional data-stacks of optical/SAR combinations. This paper outlines the results of these dense time-series classifications and how these results were affected by changing numbers of agriculture classes, numbers of available SAR imagery and numbers of training and validation data points for individual crop types. In general, for the dense time-series SAR stacks, overall accuracies of greater than 85%, a typical operational goal, were obtained for 6 of 12 sites. These results have important operational implications for particularly cloudy regions where the availability of optical imagery is limited.

Reliable, accurate and up-to-date crop type maps constitute key environmental data. Recent Copernicus satellites Sentinel-1 (S1) and Sentinel-2 (S2) with large swath widths and high temporal and spatial resolutions lead crop mapping into new era. In the framework of the SensAgri H2020 project, this work presents a new seasonal crop mapping prototype service by exploiting the synergy of S1 and S2. The experimental results are carried out on three different European test sites.

The supervised classification of satellite image time series allows obtaining reliable land cover maps over large areas. However, their quality depends on the reference datasets used for training the classifier. In remote sensing, reference data may lack of timeliness and accuracy which leads to the presence of mislabeled data degrading the classification performances. This work presents an iterative learning framework to deal with noisy instances, that can be seen as outliers. Several outlier detection strategies, based on the well-known Random Forests (RF) ensemble classifier, are proposed, evaluated quantitatively, and then compared with traditional methods. Experimental results have been carried out by using synthetic and real datasets representing annual vegetation profiles.

New high resolution Satellite Image Time Series (SITS) are becoming crucial to land cover mapping over large areas. Their high temporal resolution will allow to better depict scene dynamics. However, it will also increase the amount of data to process. The classification of these data involves therefore new challenges such as: (1) selecting the best feature set to use as input data, (2) dealing with data variability coming from landscape diversity, and (3) establishing the robustness of existing classifiers over large areas. This work aims at addressing these questions through three different studies. Experimental results are obtained by using SPOT-4 and Landsat-8 SITS.

The reconstruction of missing data in image time series due to cloud masking, among other reasons, is addressed in this paper. A new patch-based strategy based on the similarities between the spatial, spectral and temporal evolutions of the image pixel values is proposed. Experiments are carried out using Landsat-8 and SPOT-5 image time series. The proposed methodology is compared to classical temporal interpolation approaches.

Developing better agricultural monitoring capabilities based on Earth Observation data is critical for strengthening food production information and market transparency. In 2014, the European Space Agency launched the Sentinel-2 for Agriculture project which aims at preparing the exploitation of Sentinel-2 data for agriculture monitoring through the development of an open source system able to generate relevant agricultural products. In order to meet this objective, the project carried out a benchmarking exercise to identify the best algorithms that will be in this system. For each product, a minimum of five algorithms were tested over 12 sites globally distributed. This paper gives a general overview of the project and presents in detail the benchmarking.

Crop area extent estimates and crop type maps provide crucial information for agricultural monitoring and management. Remote sensing imagery in general and, more specifically, high temporal and high spatial resolution data as the ones which will be available with upcoming systems such as Sentinel-2 constitute a major asset for this kind of application. The goal of this paper is to assess to which extent state of the art supervised classification methods can be applied to high resolution multi-temporal optical imagery to produce accurate crop type maps at the global scale. Five concurrent strategies for automatic crop type map production have been selected and benchmarked using SPOT4 (Take5) and LANDSAT8 data over 12 test sites spread all over the globe. The results show that a Random Forest classifier operating on linearly temporally gap-filled images can achieve overall accuracies above 80% for most sites. The approach is fully automatic.

The exploitation of new high revisit frequency earth observations by the future Sentinel-2 satellite is clearly an important opportunity for global agricultural monitoring. In this context, the Sentinel-2Agriculture project aims at producing algorithms working on large geographical areas having different climates and different agricultural systems. In the framework of this project, the construction of a near-real-time deliverable cropland mask product has been studied here. A set of 12 selected test sites are used to benchmark the proposed method with regard to the diversity of agro-ecological context, the various landscape patterns, the different agriculture practices and the actual satellite observation conditions. The classification results yield very promising accuracies achieving around 90 % at the end of the agricultural season.

The classification of an annual time series by using data from past years is investigated in this letter. Several classification schemes based on data fusion, sparse learning, and semisupervised learning are proposed to address the problem. Numerical experiments are performed on a Moderate Resolution Imaging Spectroradiometer image time series and show that while several approaches have statistically equivalent performances, a support vector machine with I1regularization leads to a better interpretation of the results due to their inherent sparsity in the temporal domain.

Real-time monitoring of agricultural crops is increasingly important because of the involved huge economic impact. The automatic identification of crops, as early as possible during the agricultural season, is an important issue supporting agricultural policies. In this context, the objective of this article is to evaluate the possibilities of remote sensing data to identify corn and soybean crops in the early growing season. The proposed study evaluates the potential of hyperspectral data and multispectral NDVI times series. Experimental results illustrate the challenges of crop detection in early stages.

In this work, an image representation based on Binary Partition Tree is proposed for object detection in hyperspectral images. The BPT representation defines a search space for constructing a robust object identification scheme. Spatial and spectral information are integrated in order to analyze hyperspectral images with a region-based perspective. Experimental results demonstrate the good performances of this BPT-based approach.

The current increase of spatial as well as spectral resolutions of modern remote sensing sensors represents a real opportunity for many practical applications but also generates important challenges in terms of image processing. In particular, the spatial correlation between pixels and/or the spectral correlation between spectral bands of a given pixel cannot be ignored. The traditional pixel-based representation of images does not facilitate the handling of these correlations. In this paper, we discuss the interest of a particular hierarchical region-based representation of images based on binary partition tree (BPT). This representation approach is very flexible as it can be applied to any type of image. Here both optical and radar images will be discussed. Moreover, once the image representation is computed, it can be used for many different applications. Filtering, segmentation, and classification will be detailed in this paper. In all cases, the interest of the BPT representation over the classical pixel-based representation will be highlighted.

The optimal exploitation of the information provided by hyperspectral images requires the development of advanced image-processing tools. This paper proposes the construction and the processing of a new region-based hierarchical hyperspectral image representation relying on the binary partition tree (BPT). This hierarchical region-based representation can be interpreted as a set of hierarchical regions stored in a tree structure. Hence, the BPT succeeds in presenting: 1) the decomposition of the image in terms of coherent regions, and 2) the inclusion relations of the regions in the scene. Based on region-merging techniques, the BPT construction is investigated by studying the hyperspectral region models and the associated similarity metrics. Once the BPT is constructed, the fixed tree structure allows implementing efficient and advanced application-dependent techniques on it. The application-dependent processing of BPT is generally implemented through a specific pruning of the tree. In this paper, a pruning strategy is proposed and discussed in a classification context. Experimental results on various hyperspectral data sets demonstrate the interest and the good performances of the BPT representation.

In this paper, a new technique is presented to detect the river networks in simulated SWOT images. The proposed algorithm is based on a noise reduction step followed by a directional morphological filter. In this work, the speckle noise reduction has been achieved by using a Curvelet-based filter preserving the structures of interest. After the filtering task, a river contrast enhancement has been presented by using the Path-Opening filter. This morphological filtering has retained the curvilinear structures on the image independently of their orientation. Hence, the river detection has been possible by a simple thresholding on the Path-Opening result. The obtained results are evaluated using a visual inspection and a quantitative evaluation. The potential of the proposed algorithm has been evaluated by studying the robustness of the parameters.

Remote sensing is a widely-used utility in supporting multilateral environmental treaties such as the Water Framework Directive (WFD). Regarding the WFD most remote sensing applications aim on the assessment of the biochemical status of surface water, while the general detection of water networks is disregarded. Therefore, a methodology for the automatic extraction of river networks from multispectral satellite data is presented.

The work presented here proposes a new Binary Partition Tree pruning strategy aimed at the segmentation of hyperspectral images. The BPT is a region-based representation of images that involves a reduced number of elementary primitives and therefore allows to design a robust and efficient segmentation algorithm. Here, the regions contained in the BPT branches are studied by recursive spectral graph partitioning. The goal is to remove subtrees composed of nodes which are considered to be similar. To this end, affinity matrices on the tree branches are computed using a new distance-based measure depending on canonical correlations relating principal coordinates. Experimental results have demonstrated the good performances of BPT construction and pruning.

This paper discusses hierarchical region-based representation using Binary Partition Tree in the framework of hyperspectral data. Based on region merging techniques, this region-based representation reduces the number of elementary primitives compared to the pixel based representation and allows a more robust filtering, segmentation, classification or information retrieval. The work presented here proposes a strategy for merging hyperspectral regions using a new association measure depending on canonical correlations relating principal coordinates. To demonstrate an example of BPT usefulness, a pruning strategy aiming at object detection is discussed. Experimental results demonstrate the good performances of BPT.

Hyperspectral imaging segmentation has been an active research area over the past few years. Despite the growing interest, some factors such as high spectrum variability are still significant issues. In this work, we propose to deal with segmentation through the use of Binary Partition Trees (BPTs). BPTs are suggested as a new representation of hyperspectral data representation generated by a merging process. Different hyperspectral region models and similarity metrics defining the merging orders are presented and analyzed. The resulting merging sequence is stored in a BPT structure which enables image regions to be represented at different resolution levels. The segmentation is performed through an intelligent pruning of the BPT, that selects regions to form the final partition. Experimental results on two hyperspectral data sets have allowed us to compare different merging orders and pruning strategies demonstrating the encouraging performances of BPT-based representation.

The optimal exploitation of the information provided by hyperspectral images requires the development of advanced image processing tools. This paper introduces a new hierarchical structure representation for such images using binary partition trees (BPT). Based on region merging techniques using statistical measures, this region-based representation reduces the number of elementary primitives and allows a more robust filtering, segmentation, classification or information retrieval. To demonstrate BPT capabilities, we first discuss the construction of BPT in the specific framework of hyperspectral data. We then propose a pruning strategy in order to perform a classification. Labelling each BPT node with SVM classifiers outputs, a pruning decision based on an impurity measure is addressed. Experimental results on two different hyperspectral data sets have demonstrated the good performances of a BPT-based representation.

This paper presents a new method for extracting roads in Very High Resolution remotely sensed images based on advanced directional morphological operators. The proposed approach introduces the use of Path Openings and Closings in order to extract structural pixel information. These morphological operators remain flexible enough to fit rectilinear and slightly curved structures since they do not depend on the choice of a structural element shape and hence outperform standard approaches using rotating rectangular structuring elements. The method consists in building a granulometry chain using Path Openings and Closing to perform Morphological Profiles. For each pixel, the Morphological Profile constitutes the feature vector on which our road extraction is based.

In the frame of seismic vulnerability assessment in urban areas, it is very important to estimate the nature of the roof of every building and, in particular, to make the difference between flat roofs and gable ones. In order to perform this tedious task automatically on a large scale, remote sensing data provide a useful solution. In this study, we use simultaneously very high resolution panchromatic data, and an accurate digital elevation model. The fusion of these two modalities enables the extraction of two mixed features. Based on these features the classification between the two considered classes becomes a simple linearly separable problem.