Although recently several foundation models for satellite remote sensing imagery have been proposed, they fail to address major challenges of operational applications. Indeed, representations that do not take into account the spectral, spatial and temporal dimensions of the data as well as the irregular or unaligned temporal sampling are of little use for most real world applications. As a consequence, we address some existing shortcomings in the design of foundation models for remote sensing data. In particular, we propose an ALIgned Sits Encoder (ALISE), a novel approach that leverages the spatial, spectral, and temporal dimensions of irregular and unaligned Satellite Image Time Series (SITS), while producing aligned latent representations. ALISE provides easy-to-use fixed-size SITS representations which preserve the spatial resolution of the input SITS required for most mapping tasks. Moreover, to learn informative representations of SITS, we investigate the integration of instance discrimination losses within a masked auto-encoding pre-training task, utilizing a multi-view framework. The model is pre-trained on a custom-built Sentinel-2 multi-year SITS unlabeled data-set. The genericity of the provided representations is assessed on three downstream tasks: crop segmentation, land cover segmentation, and an unsupervised crop change detection task. The results suggest that the use of ALISE’s aligned representations is significantly more effective than previous SSL methods for linear probing segmentation tasks. Additionally, the experiments show the interest of using ALISE representations for unsupervised change detection. Lastly, the impact of the pre-training hyperparameters and the proposed method for aligning irregular and unaligned time series are examined in detail. The code, the pre-trained model as well as the data-sets are released at https://src.koda.cnrs.fr/iris.dumeur/alise.

In remote sensing, Single Image Super-Resolution can be learned from large cross-sensor datasets with matched High Resolution and Low Resolution satellite images, thus avoiding the domain gap issue that occurs when generating the Low Resolution image by degrading the High Resolution one. Yet cross-sensor datasets come with their own challenges, caused by the radiometric and geometric discrepancies that arise from using different sensors and viewing conditions. While those discrepancies can be prominent, their impact has been vastly overlooked in the literature, which often focuses on pursuing more complex models without questioning how they can be trained and fairly evaluated in a cross-sensor setting. This paper intends to fill this gap and provide insight on how to train and evaluate cross-sensor Single-Image Super-Resolution Deep Learning models. First, it investigates standard Image Quality metrics robustness to discrepancies and highlights which ones can actually be trusted in this context. Second, it proposes a complementary set of Frequency Domain Analysis based metrics that are tailored to measure spatial frequency restoration performances. Metrics tailored for measuring radiometric and geometric distortion are also proposed. Third, a robust training and evaluation strategy is proposed, with respect to discrepancies. The effectiveness of the proposed strategy is demonstrated by experiments using two widely used cross-sensor datasets: Sen2Venμs and Worldstrat. Those experiments also showcase how the proposed set of metrics can be used to achieve a fair comparison of different models in a cross-sensor setting. The code will be publicly available at https: //github.com/Evoland-Land-Monitoring-Evolution/sisr4rs.git.

In this paper, a new self-supervised strategy for learning meaningful representations of complex optical Satellite Image Time Series (SITS) is presented. The methodology proposed named U-BARN, a Unet-BERT spAtio-temporal Representation eNcoder, exploits irregularly sampled SITS. The designed architecture allows learning rich and discriminative features from unlabelled data, enhancing the synergy between spatio-spectral and temporal dimensions. To train on unlabelled data, a time series reconstruction pretext task inspired by the BERT strategy is proposed. A Sentinel-2 large-scale unlabelled dataset is used to pre-trained U-BARN. To demonstrate its feature learning capability, representations of SITS encoded by U-BARN, are then used to generate semantic segmentation maps. Experimental results, on a labelled PASTIS dataset, corroborate that accuracies obtained by a shallow classifier using representations learned by the pre-trained model are better than results obtained by the raw SITS. Additionally, a fully supervised experiment is conducted on this same labelled PASTIS dataset to evaluate the effectiveness of the proposed U-BARN architecture. The obtained results show that U-BARN architecture reaches performances similar to the spatio-temporal baseline (U-TAE).

Multi-decade time series of the snow cover area are typically derived from low resolution sensors such as MODIS (20 years, 500 m) or AVHRR (35 years, 1 km) and fail to capture the high spatial variability of mountain snowpack [1] , [2] . The vast Landsat archive (Landsat 5-8), with an image of the same location captured every 16 days at 30 m spatial resolution since 1984 remains largely untapped in European mountains. In addition, the recent initiative by the French Space Agency (CNES) to release in the public domain the full collection of SPOT 1-5 images with the SPOT World Heritage (SWH) program [3] provides a unique opportunity to densify the Landsat time series from 1986 to 2015 with thousands of 20 m resolution multispectral images, [4] .

Land-use/land-cover (LULC) maps describe the Earth’s surface with discrete classes at a specific spatial resolution. The chosen classes and resolution highly depend on peculiar uses, making it mandatory to develop methods to adapt these characteristics for a large range of applications. Recently, a convolutional neural network (CNN)-based method was introduced to take into account both spatial and geographical context to translate a LULC map into another one. However, this model only works for two maps: one source and one target. Inspired by natural language translation using multiple-language models, this article explores how to translate one LULC map into several targets with distinct nomenclatures and spatial resolutions. We first propose a new data set based on six open access LULC maps to train our CNN-based encoder-decoder framework. We then apply such a framework to convert each of these six maps into each of the others using our Multi-Landcover Translation network (MLCT-Net). Extensive experiments are conducted at a country scale (namely France). The results reveal that our MLCT-Net outperforms its semantic counterparts and gives on par results with mono-LULC models when evaluated on areas similar to those used for training. Furthermore, it outperforms the mono-LULC models when applied to totally new landscapes.

In this article, we propose an approach based on Gaussian processes (GPs) for large-scale land cover pixel-based classification with Sentinel-2 satellite image time series (SITS). We used a sparse approximation of the posterior combined with variational inference to learn the GP’s parameters. We applied stochastic gradient descent and GPU computing to optimize our GP models on massive datasets. The proposed GP model can be trained with hundreds of thousands of samples, compared to a few thousands for traditional GP methods. Moreover, we included the spatial information by adding the geographic coordinates into the GP’s covariance function to efficiently exploit the spatio-spectro-temporal structure of the SITS. We ran experiments with Sentinel-2 SITS of the full year 2018 over an area of 200000 km2 (about 2 billion pixels) in the south of France, which is representative of an operational setting. Adding the spatial information significantly improved the results in terms of classification accuracy. With spatial information, GP models have an overall accuracy of 79.8. They are more than three points above random forest (the method used for current operational systems) and more than one point above a multilayer perceptron. Compared to a transformer-based model (which provides state-of-the-art results in the literature, but is not applied in operational systems), GP models are only one point below.

Recent Sentinel satellite constellations and deep learning methods offer great possibilities for estimating the states and dynamics of physical parameters on a global scale. Such parameters and their corresponding uncertainties can be retrieved by machine learning methods solving probabilistic inverse problems. Nevertheless, the scarcity of reference data to train supervised methodologies is a well-known constraint for remote sensing applications. To address such limitations, this work presents a new generic physics-guided probabilistic deep learning methodology to invert physical models. The presented methodology proposes a new strategy to combine probabilistic deep learning methods and physical models avoiding simulation-driven machine learning. The inverse problem is addressed through a Bayesian inference framework by proposing a new physically-constrained self-supervised representation learning methodology. To show the interest of the proposed strategy, the methodology is applied to the retrieval of phenological parameters from NDVI time series. As a result, the probability distributions of the intrinsic phenological model parameters are inferred. The feasibility of the method is evaluated on both simulated and real Sentinel-2 data and compared with different standard algorithms. Promising results show satisfactory accuracy predictions and low inference times for real applications.

This paper presents a framework for simultaneously translating multiple land-cover maps into a given one in a supervised way. Conversely to existing approaches working on 1–1 translation, we propose a multi-translation setup that increases the generalizability and translation performance, especially on land-cover maps covering restricted spatial extents. The proposed method mainly assumes that the map of interest spatially overlaps at least with one of the other maps. High performance translation is achieved with a Convolutional Neural Network (CNN) based encoder-decoder frame-work trained with three goals: (i) high-quality translation; (ii) self-reconstruction ability; (iii) mapping of all datasets into a common representation space. Country-scale experimental results show the method effectiveness in translating six highly heterogeneous land-cover maps, achieving significantly better results than the traditional semantic-based method and better results than CNN trained for a 1–1 translation task (+ 9.7% in Overall Accuracy (OA) and +12% in macro F1-score (mF1)).

This paper presents a framework for translating a land-cover map into another one in a supervised way. This links to numerous applications (updating, completion, etc.). Conversely to existing approaches, we jointly perform spatial and semantic transformation without any prior knowledge. The proposed method assumes that: i) examples of the source and target maps already exist, ii) the spatial resolution of the source map is equal or higher than the target one. The translation is performed using an asymmetric Convolutional Neural Network with positional encoding. Experimental results show the effectiveness of the method in retrieving a yearly version of Corine Land Cover (CLC) at country-scale (France) using an existing high-resolution map and with similar accuracy than existing CLC maps (~80%).

The growing number of images acquired by new satellite missions increases the interest of learning low dimensional image representations without human supervision. Variational AutoEncoders (VAE) are one of the most promising strategies marrying graphical models and deep learning. They are able to learn continuous, structured and probabilistic latent spaces encoding the data. In this work, a VAE architecture is proposed and analyzed for multispectral Sentinel-2 images. The regularized β-VAE is studied and compared with the classical auto-encoder strategy. A classification experiment is carried out to corroborate that generated latent spaces can preserve the salient features of the input data. Classification results show that high accuracies can be obtained by using low dimensional latent representation learned by VAE as input data in a standard classification approach.

Image segmentation techniques are challenging to apply to large-size remote sensing imagery. Indeed, if the data to be processed are larger than the computer's available memory, it must be split into smaller pieces. Without precaution, segmentation errors appear along the edges of these pieces. The goal of this paper is to present a tilewise processing method to overcome this issue for superpixel segmentation, applied in particular to the simple linear iterative clustering algorithm. Incidentally, tilewise methods allow for several pieces of the image to be processed simultaneously, which enables the deployment of these methods in a parallel processing environment. Estimations of the speed-up when using multiple processors are provided. Then, it is demonstrated that the result of the tilewise segmentation is equivalent to the segmentation of the complete image, with respect to a number of global unsupervised segmentation criteria. Finally, experimental results on a large-size Sentinel-2 time series validate the method's feasibility.

High resolution image time series as those provided by Sentinel-2 allow to target semantically rich nomenclatures for land cover mapping. However, at 10 m resolution, pixel based classification fails to correctly identify some classes for which pixel context is discriminative. Recent advances in deep convolutional neural networks show promising results to tackle this problem, but the lack of complete annotation over large areas, the computational cost and the dimensionality of the feature space (much larger than those used in computer vision) does not allow to use these approaches in operational mapping applications yet. Contextual information can be calculated by applying a fixed-size neighborhood filter, but this can cause the loss of linear objects and the rounding of sharp corners. In Object Based Image Analysis, segmentation is used to extract objects for calculating contextual features while maintaining the high-frequency elements in the image. However, these do not necessarily include spectrally diverse pixels in a neighborhood, which can be relevant for characterizing the context. Superpixels place themselves in between the fixed-neighborhood and the object-based methods, in that they include spectrally diverse pixels in the same segment by imposing size and compacity constraints, while remaining adaptive to the natural boundaries in the image. This study assesses and compares the ability of these three types of neighborhood to improve classification performance on context-dependent classes, in a high-resolution Sentinel-2 time series land cover mapping problem.

Many Earth monitoring applications use land cover maps, with increasing demands in terms of accuracy and short production delay. In this context, methods based on supervised classification of satellite image time series are often used because they allow to reach the required accuracy. However, they require a lot of reference data to be efficient. Previous works have shown that supervised classifiers trained with images and reference data of previous periods and followed by voting based fusion can achieve very good performances. However, voting approaches can lead to indecisions when there is too much disagreement between individual classifiers. This paper proposes to use frequent class transitions and the class history of each pixel to select the labels in case of tie during the voting step. Experimental results are obtained using a dataset of 7 years of image times series and reference data. The experiments show that the use of transition information notably improves the mapping accuracy.

The supervised classification of satellite image time series allows obtaining reliable land cover maps over large areas. However, their quality depends on the reference datasets used for training the classifier. In remote sensing, reference data may lack of timeliness and accuracy which leads to the presence of mislabeled data degrading the classification performances. This work presents an iterative learning framework to deal with noisy instances, that can be seen as outliers. Several outlier detection strategies, based on the well-known Random Forests (RF) ensemble classifier, are proposed, evaluated quantitatively, and then compared with traditional methods. Experimental results have been carried out by using synthetic and real datasets representing annual vegetation profiles.

New high resolution Satellite Image Time Series (SITS) are becoming crucial to land cover mapping over large areas. Their high temporal resolution will allow to better depict scene dynamics. However, it will also increase the amount of data to process. The classification of these data involves therefore new challenges such as: (1) selecting the best feature set to use as input data, (2) dealing with data variability coming from landscape diversity, and (3) establishing the robustness of existing classifiers over large areas. This work aims at addressing these questions through three different studies. Experimental results are obtained by using SPOT-4 and Landsat-8 SITS.

Developing better agricultural monitoring capabilities based on Earth Observation data is critical for strengthening food production information and market transparency. In 2014, the European Space Agency launched the Sentinel-2 for Agriculture project which aims at preparing the exploitation of Sentinel-2 data for agriculture monitoring through the development of an open source system able to generate relevant agricultural products. In order to meet this objective, the project carried out a benchmarking exercise to identify the best algorithms that will be in this system. For each product, a minimum of five algorithms were tested over 12 sites globally distributed. This paper gives a general overview of the project and presents in detail the benchmarking.

Crop area extent estimates and crop type maps provide crucial information for agricultural monitoring and management. Remote sensing imagery in general and, more specifically, high temporal and high spatial resolution data as the ones which will be available with upcoming systems such as Sentinel-2 constitute a major asset for this kind of application. The goal of this paper is to assess to which extent state of the art supervised classification methods can be applied to high resolution multi-temporal optical imagery to produce accurate crop type maps at the global scale. Five concurrent strategies for automatic crop type map production have been selected and benchmarked using SPOT4 (Take5) and LANDSAT8 data over 12 test sites spread all over the globe. The results show that a Random Forest classifier operating on linearly temporally gap-filled images can achieve overall accuracies above 80% for most sites. The approach is fully automatic.

The exploitation of new high revisit frequency earth observations by the future Sentinel-2 satellite is clearly an important opportunity for global agricultural monitoring. In this context, the Sentinel-2Agriculture project aims at producing algorithms working on large geographical areas having different climates and different agricultural systems. In the framework of this project, the construction of a near-real-time deliverable cropland mask product has been studied here. A set of 12 selected test sites are used to benchmark the proposed method with regard to the diversity of agro-ecological context, the various landscape patterns, the different agriculture practices and the actual satellite observation conditions. The classification results yield very promising accuracies achieving around 90 % at the end of the agricultural season.

Processing large very high-resolution remote sensing images on resource-constrained devices is a challenging task because of the large size of these data sets. For applications such as environmental monitoring or natural resources management, complex algorithms have to be used to extract information from the images. The memory required to store the images and the data structures of such algorithms may be very high (hundreds of gigabytes) and therefore leads to unfeasibility on commonly available computers. Segmentation algorithms constitute an essential step for the extraction of objects of interest in a scene and will be the topic of the investigation in this paper. The objective of the present work is to adapt image segmentation algorithms for large amounts of data. To overcome the memory issue, large images are usually divided into smaller image tiles, which are processed independently. Region-merging algorithms do not cope well with image tiling since artifacts are present on the tile edges in the final result due to the incoherencies of the regions across the tiles. In this paper, we propose a scalable tile-based framework for region-merging algorithms to segment large images, while ensuring identical results, with respect to processing the whole image at once. We introduce the original concept of the stability margin for a tile. It allows ensuring identical results to those obtained if the whole image had been segmented without tiling. Finally, we discuss the benefits of this framework and demonstrate the scalability of this approach by applying it to real large images.

Large scale segmentation remains a challenging task because of time and memory consuming. A usual strategy to process efficiently a large volume of data is to divide into chunks to be processed separately, either sequentially to reduce memory footprint or in parallel in order to speed up the computation. In image processing in general this boils down to dividing the input image into tiles. However, for image segmentation, the tile splitting usually leads incoherent segments on the borders of the tiles even when some overlap between the tiles is applied. In this paper we propose a new strategy making possible the tiling for image segmentation algorithms while maintaining the accuracy of the final results. Specifically, we focus on iterative region merging methods but the strategy can be extended to any segmentation algorithm. The introduction of the local mutual best fitting concept and the area of influence of a segment allows to establish a new methodology of segmentation based on three phases: the tile-based reduction, the iterative reduction and the completion of the segmentation. This new methodology was applied on a large Pleiades HR image with success proving the feasibility of the approach.

This paper presents a method for hedgerow extraction from very high resolution optical images using image segmentation. The method is based on a connected component region growing approach followed by an object based image analysis filtering allowing to introduce high level knowledge about the characteristics of the desired hedgerows. The proposed approach is used as a part of the EODHaM image processing system developed in the framework of the BIO SOS FP7 project.

The generation of land-cover maps for agriculture is a recurrent problem in remote sensing. There exist many efficient algorithms, but they often need well selected images during specific periods, which delays the map availability to the end of the season. In this work, we propose to introduce prior knowledge about crop rotation and topography in order to both improve the classification and obtain an accurate map early in the year. We use a Bayesian Network to model the crop rotation and we introduce the output of the model into a Support Vector Machine classifier to generate a land-cover map. We evaluate the overall improvement and the effect on several crops.

Frequent high-resolution images will be provided by new satellites such as Venμs, SENTINEL-2 and Landsat Data Continuity Mission. Methods to handle this new type of data are currently developed (see [1] for an example). However, a more frequent observation of the surface of the Earth may be required for some applications. Moreover, the temporal resolution may be reduced by meteorological artifacts. In this work, we propose to take advantage of the higher temporal resolution of satellites with a lower spatial resolution to detect land-cover modification at a high spatial resolution. The proposed approach does not use any fusion step of the high- and low-resolution images. We show that the low spatial resolution satellite image time series (SITS) can be used in order to inform about the stability and relevance of the high spatial resolution classification. Experiments include a wide variety of resolution ratios and study the use of each ratio for the assessment of high resolution classification maps (computed from the high spatial resolution SITS).

This work presents a methodology for the fast exploitation of the large volumes of high temporal and spectral resolution data that will be available with the future Earth Observation missions. A new approach integrating temperature and phenological information for the characterisation of land cover classes is given, as part of a fully automatic system for the generation of large area land cover maps. No selection of cloud-free dates, masking of unsuitable regions, or user interaction is needed. Analysis of its performance is undertaken, and future directions are identified.

Alignment and parallelism are frequently found between objects in high-resolution remote sensing images and can be used to interpret and describe the observed scenes. In this paper, we propose new representations of parallelism and alignment as fuzzy spatial relations, which capture the imprecision in the semantics of both relations. We propose two novel definitions of alignment between objects: local and global. In local alignment, each object of the group is aligned with its neighbors, while in global alignment, every object of the group is aligned to all other members. Both definitions consider each object as a whole and are based on relative position measures. They are robust with respect to segmentation errors. Furthermore, we propose an efficient graph-based method to determine which are the locally and the globally aligned groups of objects from a set of segmented objects. In addition, we propose a fuzzy definition for the parallel relation, which is also based on relative position measures and is adequate to represent the parallelism between a globally aligned group of objects and another object or group of objects. Illustrative examples on optical satellite images show the description power of these two relations and their combination for image interpretation.

In this paper we present a procedure for the segmentation of high resolution image time series of cropland areas. We use a Land Parcel Information System which gives us the parcel boundaries, but some of the parcels need to be split in several fields since they contain several crops. The procedure is based in a template matching approach which uses single crop parcels in order to generate reference signatures for the different crop classes and a similarity metric to match every pixel of the mixed parcel to the corresponding crop reference signature.

The generation of land-cover maps for agriculture is a recurrent problem in remote sensing. There exist many efficient algorithms, but they often need well selected images during specific periods, which delays the map availability to the end of the season. In this work, we propose to introduce prior knowledge about the crop rotation in order to both improve the classification and obtain an accurate map early in the year. We use a Bayesian Network to model the crop rotation and we introduce the output of the model into a Support Vector Machine classifier to generate a land-cover map. We evaluate the overall improvement and the effect on several crops.

Satellite Image Time Series are becoming increasingly available and will continue to do so in the coming years thanks to the launch of space missions which aim at providing a coverage of the Earth every few days with high spatial resolution. In the case of optical imagery, it will be possible to produce land use and cover change maps with detailed nomenclatures. It has been shown that the Dynamic Time Warping similarity measure is a consistent tool for the comparison of radiometric profiles of temporal evolution. Actually, it makes it possible to compare time series with both different lengths and different sampling. This property allows us to make the most of partially cloud-covered images, but also to transfer the knowledge learned on an agronomical year in order to classify the next year without using reference data. This article pursues this work on satellite image time series analysis and focuses on the introduction of constraints in the distance in order to fit to the expert's knowledge about the observed phenomena.

Efficient unsupervised production of large-area land cover maps with the volumes of data to be generated by the forthcoming Earth observation missions is challenging in terms of computation costs and data variability. As a solution, introduction of non-spectral knowledge for data reduction and selection is proposed here. Analysis of intra-strata variability and inter-strata correlation for different stratified sampling approaches is presented, and valuable variables for both stratification and classification are identified.

Nowadays, several optical space-borne systems with high resolution, high temporal revisit frequency and constant viewing angles are preparing to be be launched: Venμs, Sentinel-2, etc. The usefulness of these data will be limited due to for instance cloud coverage over the scene. Image fusion techniques with other satellite products of even higher revisit frequency will dramatically promote the usefulness of the data. Therefore, our objective is to find the proper image fusion technique to adapt these new missions. The Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) is one the techniques we implemented. During our research, this technique is modified to fit the parameters of our data, and the result shows an obvious improvement.

Satellite Image Time Series are becoming increasingly available and will continue to do so in the coming years thanks to the launch of space missions which aim at providing a coverage of the Earth every few days with high spatial resolution. In the case of optical imagery, it will be possible to produce land use and cover change maps with detailed nomenclatures. However, due to meteorological phenomena, such as clouds, these time series will become irregular in terms of temporal sampling, and one will need to compare time series with different lengths. In this paper, we present an approach to image time series analysis which is able to deal with irregularly sampled series and which also allows the comparison of pairs of time series where each element of the pair has a different number of samples. We present the dynamic time warping from a theoretical point of view and illustrate its capabilities with two applications to real-time series.

This paper presents a general framework for the simulation of remote sensing image time series with spatial, textural, spectral and temporal realistic characteristics. The main goal of this work is to be able to produce data which is representative of the kind of images which will be acquired by future space Earth observation missions as for instance VENμS, Sentinel 2 or LDCM. This simulated data will be used for time series image analysis algorithm development and validation.

In this paper, we investigate different architectures for an efficient object detection processing chain for high resolution remote sensing imagery, inspired from work in natural images where object detection has reached an almost operational state. Such a processing chain consists of several tasks, and for each of them, one or more methods are proposed in this paper: examples database, negative examples sampling, relevant features, learning and detection strategies, etc. Experimental results are presented, showing that the histogram of oriented gradient descriptor seems to be the most appropriate one for plane detection at a resolution of 70 centimeters.

Satellite Image Time Series are becoming increasingly available and will continue to do so in the coming years thanks to the launch of space missions which aim at providing a coverage of the Earth every few days with high spatial resolution. In the case of optical imagery, it will be possible to produce land use and cover change maps with detailed nomenclatures. However, due to meteorological phenomena, such as clouds, these time series will become irregular in terms of temporal sampling and one will need to compare time series with different lengths. In this paper we present an approach to image time series analysis which is able to deal with irregularly sampled series and which also allows the comparison of pairs of time series where each element of the pair has a different number of samples. We present the Dynamic Time Warping from a theoretical point of view and illustrate its capabilities for domain adaptation.

This paper addresses the issue of cartographic database (DB) creation or updating using high-resolution synthetic aperture radar and optical images. In cartographic applications, objects of interest are mainly buildings and roads. This paper proposes a processing chain to create or update building DBs. The approach is composed of two steps. First, if a DB is available, the presence of each DB object is checked in the images. Then, we verify if objects coming from an image segmentation should be included in the DB. To do those two steps, relevant features are extracted from images in the neighborhood of the considered object. The object removal/inclusion in the DB is based on a score obtained by the fusion of features in the framework of Dempster-Shafer evidence theory.

This paper studies database updating using optical and synthetic aperture radar images. Logistic regression is used to model the conditional probability of presence/absence of buildings given features extracted from the images. The logistic regression parameters are estimated using the maximum likelihood method. Binary hypothesis tests are then constructed from these estimates to detect changes between the optical/radar images and the existing database. The estimation and detection algorithms are evaluated using simulated and real data sets.

This paper discusses several ways in which a community of volunteers can be put together to generate good quality geographical information by the use of remote sensing image analysis tools. Three different scenarios are proposed and a system architecture, based on existing open source solutions, is suggested and discussed. The main objective of this paper is to motivate contributions to build such a system.

This paper presents an assessment on the impact of cloudy images in land cover map production using high temporal and spatial resolution optical remote sensing images.

This paper investigates the usefulness of approximation and randomness in linear filtering in order to decrease computation time. Pouring inspiration from Compressive Sensing techniques, we implement the convolution product operation using a fewer number of samples from the convolution kernel. Depending on the use case, either the higher values of the kernel or a random subset of them are used. Three applications of the principle are used to illustrate the approach: Gabor filters, quick-look production and disparity map estimation by linear correlation.

High resolution optical remote sensing images allow to produce accurate land-cover maps. This is usually achieved using an ad-hoc mixture of image segmentation and supervised classification. The main drawback of this approach is that it does not scale for real world complete scenes. In this paper we present a framework which allows to implement this kind of image analysis without scale issues.

This paper addresses the issue of cartographic database creation or updating using high resolution SAR and optical images. It proposes a processing chain to create or update road databases in urban environment. The approach is composed of two steps. First, if a database is available, the presence of each database object is checked in the images. Then, we verify if road hypotheses extracted from images should be included in the database. These two steps are conducted by extracting relevant features from the images in the neighborhood of the considered object. The object removal/inclusion in the database is based on a score obtained by the fusion of features in the framework of Dempster-Shafer evidence theory.

In this article we present a method for extracting groups of aligned objects from a labeled image. Our method is based on fuzzy measures of relative direction between the objects, leading to a fuzzy approach for defining alignment as a spatial relation. The method is able to capture the ambiguities presented when defining alignment between objects of different sizes. Two definitions of alignment are presented; local and global. The local alignments are first extracted and are used as candidates for the global alignments. Applications of the alignment relation on real images illustrates its interest for high level image interpretation.

Orfeo Toolbox, OTB, is a remote sensing image processing library developed by CNES, the French Space Agency. OTB is distributed as Open Source software and is therefore available for any remote sensing scientist or processing chain developer. This paper describes the main features of OTB, how it can be used and the expected evolutions in the coming months.

Satellite observation is particularly enticing due to its large acquisition capabilities. However these large capabilities kindle new challenges for information analysis. Object counting is one of those. To help releasing constraints on the human operator, it is important to free him from repetitive tasks and focus his attention on the high level tasks for which algorithms are not suitable yet. This abstract focuses on building counting in dense areas. The processing is done using the Or-feo Toolbox, an open-source image processing library. This paper proposes several methods with different trade-offs in terms of performance and user involvement. The method has been adapted and successfully used in other situations, as for instance counting tree stands or tents in a refugee camp.

The task of finding correspondences between images or objects in images is usually needed in remote sensing applications such as object recognition and image registration. To do this, interest — or salient — points can be used. These points are characteristic locations in images to which a descriptor can be associated to describe local features. The descriptors must be pertinent, robust to geometric and radiometric distortions. The ORFEO Toolbox includes innovative interest points detectors, Harris, SIFT [1] (Scale Invariant Feature Transformation) and the recently added SURF [2] (Speed Up Robust Features). The ORFEO Toolbox (OTB) provides a complete and efficient environment for developing elaborated applications thanks to the pipeline mechanism which ties successive processing steps and is able to deal with different image types. In this paper, we asses the interest point detectors available in OTB.

One of the key features requested by the users of the upcoming ORFEO system is the availability of tools for urban area monitoring. The needs go from coarse urban extension delimitation to individual building detection. In this context, several algorithms have been implemented and assessed in the ORFEO Toolbox library. In this work the implementation of three algorithms for urban area extraction and their assessment in terms of quality of the results and processing time are presented. These algorithms have been developed using the ORFEO Toolbox, OTB, and two of them are available inside the library since version 3.0 (April 2009).

This paper is built upon the feedback of the Orfeo ToolBox development team in the task of selecting, implementing and qualifying state-of-the-art image processing algorithms from the literature. It enforces the need to release reference implementations along with published materials, and insists on the benefits expected for both the authors and the scientific community. Finally, it exposes the numerous advantages of integrating such reference implementations into a rich software framework like the Orfeo ToolBox. Short algorithm implementation stories are given to support the different points.

This paper proposes simple focusing techniques to assist high resolution remote sensing imagery users in the task of object detection and recognition. Efficient algorithms in this field are very time consuming and thus hardly compatible with the size of remote sensing data from the market. Meanwhile, objects and areas of interest are usually sparsely spread over the scene. There might even be parts of the scene in which there are little or no chance to meet a given kind of object. We intend to show that with some very simple and fast rules, an astonishing amount of the image can be discarded with little or no important data loss. A priori and templates based techniques are described along with examples. The use of synoptic primal sketches to ease fast assessment of the images content is also developed.

High resolution remote sensing image segmentation is a great challenge in terms of potential applications, but also because of the difficulty of the task. Fully automatic algorithms are not able to extract all the desired features from complex images but visual image analysis is time consuming and tedious (therefore error prone). In this work we present a simple, yet powerful approach for interactive image segmentation. This approach combines the best of the automatic image processing together with the ability of a human operator to choose the objects of interest for a given application. Results are presented on a wide variety of objects and contexts.

This paper addresses the issue of cartographic database creation or update using high resolution SAR and optical images. In cartographic applications, objects of interest are mainly buildings and roads. This paper proposes a processing chain to update building databases. The approach is composed of two steps. First, the presence of each database object is checked in the images. Then, we verify if objects coming from an image segmentation should be added in the database. To do those two steps, features are extracted from images in the neighborhood of the considered object. The object removal/inclusion in the database is based on a score obtained by the fusion of features in the framework of Dempster Shafer evidence theory.

This paper addresses the problem of defining a similarity measure between an observed Gaussian image and a binary image constructed from a cartographic database. The main idea is to assume that the binary image has been obtained by thresholding an unobserved Gaussian image correlated with the observed image. The proposed statistical model is then used to estimate its unknown parameters using the maximum likelihood method. The paper discusses a possible application to change detection between a cartographic vector data base and an optical image.

High resolution remote sensing (HR RS) images allow discriminating between different objects in a scene. Spatial reasoning techniques can be used to interpret and describe the scene. One component of spatial reasoning deals with the modeling and assessment of spatial relations among objects. In this work we propose three models that seize the semantics of the spatial relations "to go across" and "to go through" between a linear object and a region. To develop these three models we considered the usual perception of these natural language expressions which leads to the development of the three fuzzy models. They have been implemented and tested in scenes of HR RS images. Results are in good agreement with intuition.

Satellite imagery classification using the support vector machine (SVM) algorithm may be a time-consuming task. This may lead to unacceptable performances for risk management applications that are very time constrained. Hence, methods for accelerating the SVM classification are mandatory. From the SVM decision function, it can be noted that the classification time is proportional to the number of support vectors (SVs) in the nonlinear case. In this letter, four different algorithms for reducing the number of SVs are proposed. The algorithms have been tested in the frame of a change detection application, which corresponds to a change-versus-no-change classification problem, based on a set of generic change criteria extracted from different combinations of remote sensing imagery.

High-resolution (HR) remote-sensing images allow us to access new kinds of information. Classical techniques for image analysis, such as pixel-based classifications or region-based segmentations, do not allow to fully exploit the richness of this kind of images. Indeed, for many applications, we are interested in complex objects which can only be identified and analyzed by studying the relationships between the elementary objects which compose them. In this paper, the use of a spatial reasoning technique called region connection calculus for the analysis of HR remote-sensing images is presented. A graph-based representation of the spatial relationships between the regions of an image is used within a graph-matching procedure in order to implement an object detection algorithm.

The 2008 Data Fusion Contest organized by the IEEE Geoscience and Remote Sensing Data Fusion Technical Committee deals with the classification of high-resolution hyperspectral data from an urban area. Unlike in the previous issues of the contest, the goal was not only to identify the best algorithm but also to provide a collaborative effort: The decision fusion of the best individual algorithms was aiming at further improving the classification performances, and the best algorithms were ranked according to their relative contribution to the decision fusion. This paper presents the five awarded algorithms and the conclusions of the contest, stressing the importance of decision fusion, dimension reduction, and supervised classification methods, such as neural networks and support vector machines.

Due to their flexibility, and capacity to handle high dimensional vectorial data, support vector machines (SVMs) have become the reference for remote sensing imagery classification. However when processing large amounts of data the SVM classification could be a time consuming process. In this paper a new decomposition scheme of the SVM decision function is proposed. The decomposition is based on using the Taylor series expansion to approximate the kernel function. Then, using the results of the optimization problem of the SVM after the learning phase, this expansion is used to obtain an approximate decision function that provides a trade-off between the classification accuracy and the processing time. This speeds-up the SVM classification if limited processing time is available and favors accuracy if sufficient processing time is available.

The problem of focusing on the most relevant information in a potentially overwhelming quantity of data has become increasingly important. Using irrelevant or noisy features not only can affect the accuracy of the classification results obtained but also the convergence time. In this paper several feature selection algorithms used with the Support Vector Machine (SVM) algorithm are presented. The feature selection algorithms are classified as filter and wrapper approaches. Two different wrapper techniques are presented: the first one uses the generalization error estimate of the leave-one-example-out error, while the second one uses the error estimate of the leave-one-feature-out error. Filter approaches with 4 different parameters are presented, namely: the mutual information, the FScore, and two advanced entropy measures are studied. Results in the context of change detection using satellite imagery are then discussed.

In the context of change detection and due to the multitude of change scenarios, the objective is to build a generic change detection system. For many technical and operational reasons the Support Vector Machines (SVM) algorithm is used. One of the crucial steps when using the SVM algorithm is the choice of the kernel function. With the lack of a priori information the choice of the kernel function may be difficult for the user. In this paper several techniques for constructing a suitable kernel function obtained from the data are proposed.

Disparity map estimation consists in finding the geometric deformation between two images of the same scene usually acquired with different viewing angles. The disparity map can be used for 3D information extraction or for image registration. Depending on the type of deformation between images, the physics of the sensors and the accuracy needed in the application, different approaches can be applied. This paper presents a generic framework for disparity map estimation in which different modules can be combined and tuned in order to suit the constraints of different applications.

Bi-date change detection and image registration are based on local similarity measures. When applied to Synthetic Aperture Radar (SAR) observations, the first one is based on the comparison of the local probability density functions (pdf) of the 2 images, while the latter is based on correlation or mutual information (MI) measure. A specific implementation of MI has been found to be efficient in change detection from heterogeneous SAR images as well as for SAR image registration. This implementation may be splitted in 2 terms and that can be linked to the change detection part and to the registration one respectively. Hence a set of measures are proposed in order to perform similarity measure for change detection in homogeneous and heterogeneous SAR images that prevents from misregistration errors. This point of view has been applied to a set of Radarsat images and a pair or ERS images acquired in the frame of the International Charter Space and Major Disasters, corresponding to a lava flow and a flooding event.

In a previous work, we presented an approach for object recognition based on a graph matching technique using region connection calculus on the result of a multiscale segmentation. In this work we present two main improvements to the existing algorithm. First of all, all connection calculus is implemented using vector descriptions of the segmented regions. This allows a dramatic decrease in computation time. The second improvement in the system consists in modifying the multiscale segmentation approach which is now based on greylevel geodesical morphology.

With the recent (or in the very next future) availability of high resolution optical and radar satellite sensors, the need of multi-sensor image processing systems able to assist human operators in scene interpretation is more and more crucial. This paper focuses on remote sensing image understanding with exogenous data, in the framework of cartographic applications. We propose a processing chain for cartographic database creation/update using high resolution (metric and submetric) optical and/or radar remote sensing images.

This paper studies a family of distributions constructed from multivariate gamma distributions to model the statistical properties of multisensor synthetic aperture radar (SAR) images. These distributions referred to as multisensor multivariate gamma distributions (MuMGDs) are potentially interesting for detecting changes in SAR images acquired by different sensors having different numbers of looks. The first part of this paper compares different estimators for the parameters of MuMGDs. These estimators are based on the maximum likelihood principle, the method of inference function for margins, and the method of moments. The second part of the paper studies change detection algorithms based on the estimated correlation coefficient of MuMGDs. Simulation results conducted on synthetic and real data illustrate the performance of these change detectors.

Automatic road extraction is a critical feature for an efficient use of remote sensing imagery in most contexts. This paper proposes a robust geometric method to provide a first step extraction level. These results can be used as an initialization for other algorithms or as a starting point for manual road extraction. Results of the extraction are vectorized for GIS integration and for a better interaction with human experts that can refine the results. The algorithm is fast, has very few parameters and is only slightly affected by the image properties (resolution, noise). The algorithm is available in the open-source Orfeo toolbox.

Change detection of remotely sensed images is a particularly challenging task when the available data come from different sensors. Indeed, many change indicators are based on radiometry measures, operating on their differences or ratios, that are no longer reliable when the data have been acquired by different instruments. For this reason, it is interesting to study the performance of those indicators that do not rely completely on radiometric values. A series of similarity measures for automatic change detection has been investigated and their general performance compared using optical and SAR images covering a period of about six years. We could observe that the considered change detection algorithms perform differently but that none of them permits an "absolute" measure of the changes independent of the sensor. Also the dimensions of the windows, for the estimation of the pixel statistics and of the similarity measure, affect the final results.

In the framework of the International Charter "Space and Major Disasters", charter calls are made to the signing parties every time a natural or technological hazard occurs. Consequently, space data are provided by the partners in order to help local authorities to assess the damages, organize and optimize the use of available resources. In such cases, abrupt change detection algorithms are required and numerous methods have been proposed by the geoscience and remote sensing (GRS) community. In this paper statistics measured on charter calls are compared to statistics measured on change detection methods published in the literature. These statistics aim to give the image processing community a better understanding of the needs and challenges faced in the case of real life disaster scenarios.

High resolution remote sensing images allow us to access new kinds of information. Classical techniques for image analysis, such as pixel-based classifications or region-based segmentations, do not allow to fully exploit the richness of this kind of images. Indeed, for many applications, we are interested in complex objects which can only be identified and analysed by studying the relationships between the elementary objects which compose them. In this paper, the use of a spatial reasoning technique called region connection calculus for the analysis of high resolution remote sensing images will be presented.

In this paper, we present a new similarity measure for automatic change detection in multitemporal synthetic aperture radar images. This measure is based on the evolution of the local statistics of the image between two dates. The local statistics are estimated by using a cumulant-based series expansion, which approximates probability density functions in the neighborhood of each pixel in the image. The degree of evolution of the local statistics is measured using the Kullback-Leibler divergence. An analytical expression for this detector is given, allowing a simple computation which depends on the four first statistical moments of the pixels inside the analysis window only. The proposed change indicator is compared to the classical mean ratio detector and also to other model-based approaches. Tests on the simulated and real data show that our detector outperforms all the others. The fast computation of the proposed detector allows a multiscale approach in the change detection for operational use. The so-called multiscale change profile (MCP) is introduced to yield change information on a wide range of scales and to better characterize the appropriate scale. Two simple yet useful examples of applications show that the MCP allows the design of change indicators, which provide better results than a monoscale analysis

Subpixel accuracy image registration is needed for applications such as digital elevation model extraction, change detection, pan-sharpening, and data fusion. In order to achieve this accuracy, the deformation between the two images to be registered is usually modeled by a displacement vector field which can be estimated by measuring rigid local shifts for each pixel in the image. In order to measure subpixel shifts, one uses image resampling. Sampling theory says that, if a continuous signal has been sampled according to the Nyquist criterion, a perfect continuous reconstruction can be obtained from the sampled version. Therefore, a shifted version of a sampled signal can be obtained by interpolation and resampling with a shifted origin. Since only a sampled version of the shifted signal is needed, the reconstruction needs only to be performed for the new positions of the samples, so the whole procedure comes to computing the value of the signal for the new sample positions. In the case of image registration, the similarity between the reference image and the shifted versions of the image to be registered is measured, assuming that the maximum of similarity determines the most likely shift. The image interpolation step is thus performed a high number of times during the similarity optimization procedure. In order to reduce the computation cost, approximate interpolations are performed. Approximate interpolators will introduce errors in the resampled image which may induce errors in the similarity measure and therefore produce errors in the estimated shifts. In this paper, it is shown that the interpolation has a smoothing effect which depends of the applied shift. This means that, in the case of noisy images, the interpolation has a denoising effect, and therefore, it increases the quality of the similarity estimation. Since this blurring is not the same for every shift, the similarity may be low for a null shift (no blurring) and higher for shifts close to half a pixel (strong blurring). This paper presents an analysis of the behavior of the different interpolators and their effects on the similarity measures. This analysis will be done for the two similarity measures: the correlation coefficient and the mutual information. Finally, a strategy to attenuate the interpolation artifacts is proposed

This paper evaluates the potential interest of using bivariate gamma distributions for image registration and change detection. The first part of this paper studies estimators for the parameters of bivariate gamma distributions based on the maximum likelihood principle and the method of moments. The performance of both methods are compared in terms of estimated mean square errors and theoretical asymptotic variances. The mutual information is a classical similarity measure which can be used for image registration or change detection. The second part of the paper studies some properties of the mutual information for bivariate Gamma distributions. Image registration and change detection techniques based on bivariate gamma distributions are finally investigated. Simulation results conducted on synthetic and real data are very encouraging. Bivariate gamma distributions are good candidates allowing us to develop new image registration algorithms and new change detectors.

In this paper, we present a new similarity measure for automatic change detection in multitemporal SAR images. This measure is based on the evolution of the local statistics of the image between two dates. The local statistics are estimated using a cumulant-based series expansion which approximates the probability density functions in the neighborhood of each image pixel. The degree of evolution of the local statistics is measured using the Kullback-Leibler divergence. An analytical expression for this detector is given allowing a simple computation which depends only on the 4 first statistical moments of the pixels inside the analysis window. The concept of multiscale change profile (MCP) is also introduced and its optimized implementation is presented. MCP yields change information on a wide range of scales and better characterizes the appropriate scale to be used for the detection. Two simple examples of application show that the MCP allows the design of change indicators which provide better results than a monoscale analysis.

This paper shows how to obtain a binary change map from similarity measures of the local statistics of images before and after a disaster. The decision process is achieved by the use of a zz-SVM in which a stochastic kernel has been defined. Stochastic kernel includes two similarity measures, based on the local statistics, to detect changes from the images: 1) A distance between maginal probability density functions (pdfs) and 2) the mutual information between the two observations. Distance between marginal pdfs is evaluated by using a series expansion of the Kullbak-Leibler distance. It is achieved by estimating cumulants up to order 4 from a sliding window of fixed size. Mutual information is estimated through a parametric model that is issued from the copulas theory. It is based on rank statistics and yields an analytic expression, that depends on the parameter of the copula only, to be evaluated to obtain the mutual information. Preliminary results are shown on a pair of Radarsat images acquire before and after a lava flow. A ground truth allows to show the accuracy of the stochastic kernels and the SVM decision.

In this letter, we show that the polarimetric behavior of point targets is preserved even in the case of multipolarization incoherent acquisitions. Point targets are defined as targets embedded in one image pixel and presenting a very stable backscatter during the integration time. We discuss in particular how the polarimetric response restoration can be helpful for point target detection and analysis from such acquisitions (e.g., ASAR Alternate Polarization mode of ENVISAT), but also for permanent scatterers interferometry applications.

With the advent of commercial satellite sensors producing images with resolutions better than 5 m., it is now possible to recognize man-made objects which were not visible at lower resolutions. In this paper we extend the work presented by Inglada and Giros (2004) to the use of high-level geometric descriptions and pre-conscious vision.

One of the applications where remote sensing could be very useful is the management of major disasters. While remote sensing has shown its interest for recovery and inventory tasks after the crisis period, an assessment of its usefulness during the crisis period is needed. Periodic image acquisitions over any point of the Earth surface, with improved resolutions available today seem to fulfill the required specifications of a global monitoring system. Earth Observation satellites in orbit today were not designed for such a purpose. However, several initiatives have been proposed in order to use them in this kind of applications, as for instance, the International Charter Space and Major Disasters, or the CEOS Disaster Management Support Group. In this paper we discuss, based on past experiences, what are the real capabilities of present and near future satellites, which are their drawbacks and how they could be used at best for real cases of crisis management. A list of recommendations with regards to what could be improved at the system level (sensor, acquisition scheduling, ground segment data production) and the techniques for information extraction (image processing, sensor fusion), is given

With the advent of commercial satellite sensors producing images with resolutions better than 5 m., it is now possible to recognize man-made objects which were not visible at lower resolutions. In this work, an image processing chain for the detection of man-made objects in high resolution remote sensing images are presented. Detection is understood as finding the smallest rectangular area in the image containing the object. These algorithms are based on learning methods and on an example data base which contains 10 classes of objects

Multisensor image registration is needed in a large number of applications of remote sensing imagery. The accuracy achieved with usual methods (manual control points extraction, estimation of an analytical deformation model) is not satisfactory for many applications where a subpixel accuracy for each pixel of the image is needed (change detection or image fusion, for instance). Unfortunately, there are few works in the literature about the fine registration of multisensor images and even less about the extension of approaches similar to those based on fine correlation for the case of monomodal imagery. In this paper, we analyze the problem of the automatic multisensor image registration and we introduce similarity measures which can replace the correlation coefficient in a deformation map estimation scheme. We show an example where the deformation map between a radar image and an optical one is fully automatically estimated.

For ten years, a long history of data was acquired by the SAR sensors on the satellite ERS-1 and ERS-2 offering a wide range of interferometric applications. In 2002, the more advanced satellite ENVISAT was launched. The SAR on board on ENVISAT (ASAR) can continue the success of the remote sensing mission of the ERS satellites and preserve or even increase the value of the archived ERS data. The subject of this study is to demonstrate the continuity of the interferometric measurements by the combination of the SAR scene of the different sensors to interferograms (cross interferometry).

Presents a method for performing change detection using a pair of SAR images acquired at different dates. The main difficulty with SAR images is the presence of speckle noise which may produce noisy change images if they are acquired with slightly different angles. The technique proposed in the present paper uses a parametric estimation of the probability distributions locally in each image as a characterization of the surfaces. The change is measured as a distance between these probability laws. The dissimilarity measure between the statistical distributions used here is a symmetric version of the Kullback-Leibler divergence.

The International Charter Space and Major Disasters was triggered by the Belgian Civil Protection Authorities after the Nyragongo volcanic eruption which took place on January 2002. This paper describes the chronology of the events and the image processing con- ducted in order to produce damage maps.

After the dramatic flash floods, the French Civil Defence triggered the International Charter "Space and Major Disasters" on the 9/sup th/ of September 2002. Images from the different satellites of the SPOT constellation were exploited in order to provide flood extent and flood impact products. The results highlight the benefits of the SWIR channel and the VHR SPOT sensors and these over a large swath.

In this paper we will introduce several similarity measures between multisensor images. These measures are based on concepts such as statistical dependence or mutual information. The use of these measures allows for the design of image registration algorithms and automatic change detection techniques.

Introduces the concepts of blind source separation and independent component analysis. We show how they can be applied to the automatic estimation of ground subsidence and atmospheric disturbances on a series of interferograms. Finally we show some results obtained on realistic simulated data which demonstrate the advantages of this approach.