Semi-supervised domain adaptation aims to transfer knowledge from a labeled source domain to a scarcely labeled target domain, despite distribution shifts. The challenge becomes greater when source and target data differ in acquisition modality, as in remote sensing where variations in sensor type (e.g., optical vs. radar), spectral properties (e.g., RGB vs. multispectral), or spatial resolution are common. This challenging scenario, known as Semi-Supervised Heterogeneous Domain Adaptation (SSHDA), requires learning across modalities with limited target labels. In this work, we propose SAHARA (Semi-supervised Adaptation in Heterogeneous domains via conditional Adversarial Representation disentanglement and Adaptive pseudo-labeling), a new method for SSHDA that combines conditional adversarial feature adaptation with dynamic pseudo-labeling to learn domain-invariant features and handle extremely scarce target annotations. Experiments on two heterogeneous remote sensing benchmarks for scene classification, conducted with both convolutional and transformer-based backbones, demonstrate that SAHARA consistently outperforms existing SSHDA and semi-supervised methods. The code is available at https: //TO-BE-DISCLOSED-UPON-ACCEPTANCE.

With the advent of the Sentinel-2 mission and its high revisit frequency, high-resolution time series of optical images, the use of satellite image time series for automatic land cover mapping has fostered. However, one of the main limitations related to this kind of imagery is the presence of clouds, which often hinders its descriptive potential by reducing the actual temporal resolution. Although some common practices exist to enable their use in land cover processing chains, the majority of them aims at reconstructing the time series upstream to the classification task, hence introducing a heavy, error-prone pre-processing step. With the aim of exploiting the capacity of deep learning networks to adaptively combine tasks, in this preliminary study we propose an end-to-end framework that simultaneously perform cloud removal and classification of a Sentinel-2 image time series for the downstream task of land cover mapping. The proposed framework is evaluated over an agricultural area in Northern Benin. Our first results show comparable performances with respect to using state-of-the-art gap filling pre-processing on Sentinel-2 time series, hence motivating further exploration.

With the huge variety of earth observation satellite missions available nowadays, the collection of multi-sensor remote sensing information depicting the same geographical area has become systematic in practice, paving the way to the further breakthroughs in automatic land cover mapping with the aim to support decision makers in a variety of land management applications. In this context, along with the increase in the volume of data available, the availability of ground truth data to train supervised models, which is usually time-consuming and costly, may even be more critical. In this scenario, the possibility to transfer a model learnt on a particular time span (source domain) to a different period of time (target domain), over the same geographical area, can be advantageous in terms of both cost and time efforts. However, such model transfer is challenging due to different climate, weather or environmental conditions affecting remote sensing data collected at different time periods, resulting in possible distribution shifts between thesourceandtargetdomains. With the aim to cope with the multi-sensor temporal transfer scenario in the context of land cover mapping, where multi-temporal and multi-scale information are used jointly, we proposeM3SPADA(Multi-sensor, Multi-temporal and Multi-scale SPatially-Aware Domain Adaptation framework), a deep learning methodology that jointly exploits self-training and adversarial learning to transfer a multi-sensor land cover classifier from a time period (year) to a different one on the same geographical area. Here, we consider the case in which each domain (source and target) is described by a pair of remote sensing data sets: a satellite image time series (SITS) of optical images and a single Very High spatial Resolution (VHR) scene. Experimental evaluation on a real-world study case located in Burkina Faso and characterized by operational constraints shows the quality of our proposal to deal with the temporal multi-sensor transfer in the context of land cover mapping.

Nowadays, Satellite Image Time Series (SITS) are employed as input to derive land cover maps (LCM) to support decision makers in several application domains like agriculture and biodiversity. The generation of LCM largely relies on available ground truth (GT) data to calibrate supervised ma-chine learning models. Unfortunately, this data are not always accessible. In this scenario, the possibility to transfer a model learnt on a particular year (source domain) to another period of time (target domain) could be a valuable tool to deal with the previously mentioned restrictions. In this paper, we provide an experimental evaluation of recent Unsupervised Domain Adaptation (UDA) methods in the specific context of temporal transfer learning for SITS-based LCM. The objective is to learn a classification model at a certain year (exploiting available GT data) and, successively, transfer such a model on a subsequent year where no labelled samples are accessible. The obtained findings reveal that UDA methods represent a promising research direction to cope with the problem of temporal transfer learning for LCM. While a model learnt on the source data and directly applied on target data achieves an weighted F1-score of 67.1, the best UDA method obtains an F1-score of 83.7 with more than 15 points of positive gap. Nevertheless, there is still room for improvement that should be explored in future works.

The Global Ecosystem Dynamics Investigation (GEDI) instrument, as all FW systems, relies on very sophisticated pre-processing steps to generate a priori metrics in order to accurately estimate forest characteristics, such as forest heights and wood volume. The ever-expanding volume of acquired GEDI data, which to September 2020 comprised more than 25 billion shots, and requiring more than 90 TB of storage space, raises new challenges in terms of adapted preprocessing methods for the suitable exploitation of such a huge and complex amount of LiDAR data. Therefore, to avoid metric computation, we leveraged deep learning techniques in order to estimate canopy dominant heights (Hdom) and wood volume (V) of Eucalyptus plantations over five different regions in Brazil. Performance comparisons were conducted between a convolutional neural network based model that uses GEDI waveform data and a previously used, metric based, Random Forest regressor (RF). Cross-validated results showed that the CNN based model compared well against the RF counterpart for both Hdomand V. Indeed, the RMSE on the estimation of Hdomfrom the CNN based model was 1.61 m with a coefficient of determination R2of 0.90, while the RF model produced an accuracy on Hdomestimates of 1.45 m(R2=0.92). For V, CNN based estimates was 27.35 m3.ha-1(R2of 0.88), while for RF, the RMSE was 27.60 m3.ha-1 (R2=0.88).

A major issue affecting optical imagery is the presence of clouds. The need of cloud-free scenes at specific date is crucial in a number of operational monitoring applications. On the other hand, the cloud-insensitive SAR sensors are a solid asset and they provide orthogonal information with respect to optical satellite, that enable the retrieval of information lost in optical images due to cloud cover. In the context of an increasing availability of both optical and SAR images, thank to the Sentinel constellation, we propose a deep learning method to reconstruct (gap-fill) optical data, polluted by cloud phenomena, exploiting multi-temporal SAR and optical images.

Radar and Optical Satellite Image Time Series (SITS) are sources of information that are commonly employed to monitor earth surfaces for tasks related to ecology, agriculture, mobility, land management planning and land cover monitoring. Many studies have been conducted using one of the two sources, but how to smartly combine the complementary information provided by radar and optical SITS is still an open challenge. In this context, we propose a new neural architecture for the combination of Sentinel-1 (S1) and Sentinel-2 (S2) imagery at object level, applied to a real-world land cover classification task. Experiments carried out on the Reunion Island, a overseas department of France in the Indian Ocean, demonstrate the significance of our proposal.

Sentinel-2 data represent a rich source of information for the community due to the free access and to the temporal-spatial coverage assured. However, some of the spectral bands are sensed at reduced resolution due to a compromise between technological limitations and Copernicus program's objectives. For this reason in this work we present a new super-resolution method based on Convolutional Neural Networks (CNNs) to rise the resolution of the short wave infra-red (SWIR) band from 20 to 10 meters, that is the highest resolution provided. This is accomplished by fusing the target band with the finer-resolution ones. The proposed solution compares favourably against several alternative methods according to different quality indexes. In addition we have also tested the use of the super-resolved band from an applicative perspective by detecting water basins through the Modified Normalized Difference Water Index (MNDWI).

Since optical remote sensing images are useless in cloudy conditions, a possible alternative is to resort to synthetic aperture radar (SAR) images. However, many conventional techniques for Earth monitoring applications require specific spectral features which are defined only for multispectral data. For this reason, in this work we propose to estimate missing spectral features through data fusion and deep learning, exploiting both temporal and cross-sensor dependencies on Sentinel-1 and Sentinel-2 time-series. The proposed approach, validated focusing on the estimation of the normalized difference vegetation index (NDVI), shows very interesting results with a large performance gain over the linear regression approach according to several accuracy indicators.

Mapping winter vegetation quality is a challenging problem in remote sensing. This is due to cloud coverage in winter periods, leading to a more intensive use of radar rather than optical images. The aim of this letter is to provide a better understanding of the capabilities of Sentinel-1 radar images for winter vegetation quality mapping through the use of deep learning techniques. Analysis is carried out on a multitemporal Sentinel-1 data over an area around Charentes-Maritimes, France. This data set was processed in order to produce an intensity radar data stack from October 2016 to February 2017. Two deep recurrent neural network (RNN)-based classifiers were employed. Our work revealed that the results of the proposed RNN models clearly outperformed classical machine learning approaches (support vector machine and random forest).

This work deals with the fusion of SAR and optical data for land cover monitoring. We first propose to use co-registered optical data as a guide for nonlocal SAR image despeckling. Then, we fuse filtered Sentinel-1 SAR data with optical Sentinel-2 data for land-use classification. Experiments show that using optical-driven despeckled SAR data largely improves classification accuracy w.r.t using the original data and even multitemporal filtered data.

Nowadays, modern earth observation programs produce huge volumes of satellite images time series that can be useful to monitor geographical areas through time. How to efficiently analyze such a kind of information is still an open question in the remote sensing field. Recently, deep learning methods proved suitable to deal with remote sensing data mainly for scene classification(i.e., convolutional neural networks on single images) while only very few studies exist involving temporal deep learning approaches [i.e., recurrent neural networks (RNNs)] to deal with remote sensing time series. In this letter, we evaluate the ability of RNNs, in particular, the long short-term memory (LSTM) model, to perform land cover classification considering multitemporal spatial data derived from a time series of satellite images. We carried out experiments on two different data sets considering both pixel-based and object-based classifications. The obtained results show that RNNs are competitive compared with the state-of-the-art classifiers, and may outperform classical approaches in the presence of low represented and/or highly mixed classes. We also show that the alternative feature representation generated by LSTM can improve the performances of standard classifiers.

In this paper a new object-oriented segmentation method for high-resolution remote sensing images is proposed. To limit computational complexity, a preliminary superpixel representation of the image is obtained by means of a suitable watershed transform. Then, a region adjacency graph is associated with the superpixels, with edge weights accounting for region similarity/dissimilarity. The final segmentation is then obtained by means of a graph-cutting approach, following a correlation clustering formulation. The optimal cut can be obtained by solving a Integer Linear Programming (ILP) problem, whose complexity, however, grows rapidly with the image size. Much faster near-optimal solutions are obtained, here, with a greedy solution. Experiments on a real-world high-resolution remote sensing image prove the potential of the approach.

We propose an interactive tool for designing ground-truth maps associated with multi-resolution remote sensing images. The target image is first segmented at object level by means of an edge-preserving algorithm. Then, a pre-classification defines groups of segments that are homogeneous both in spectral response and size. Finally, suitable candidate segments are selected and shown to the supervisor for inspection and labeling or possible rejection, in an iterative process, until the desired image covering is reached. Experimental results show that the proposed solution allows one to easily and quickly obtain ground-truth maps which are both locally and globally accurate, and where all classes are represented in a balanced manner.

We propose a new synthetic aperture radar (SAR) despeckling technique based on nonlocal filtering and driven by a coregistered optical image. A preliminary homogeneous versus heterogeneous classification of the image is used to decide where the optical guide can be safely used, thus preventing any distortion of the SAR geometry. Even in regions where the use of optical data is enabled, despeckling is carried out exclusively in the SAR domain, and the optical guide is used only to improve the predictor selection in nonlocal filtering and, hence, in the estimation process. Experiments on real-world imagery confirm the potential of the proposed approach.

A new technique for the segmentation of single- and multiresolution (MR) remote sensing images is proposed. To guarantee the preservation of details at fine scales, edge-based watershed is used, with automatically generated markers that help in limiting oversegmentation. For MR images, the panchromatic and multispectral components are processed independently, extracting both the edge maps and the morphological and spectral markers that are eventually fused at the highest resolution, thus avoiding any information loss induced by pansharpening. Numerical results on object layer extraction and simple classification tasks prove the proposed techniques to provide accurate segmentation maps, which preserve fine details and, contrary to state-of-the-art products, can single out objects equally well at very different scales.

Reliable segmentation of SAR images requires some forms of user supervision: we resort here to the interactive version of the Tree-Structured Markov Random Field (TS-MRF) segmentation suite. The TS-MRF model, and the associated segmentation tool, provide a flexible and spatially adaptive description of the data. In the interactive version, the user can drive the process based on the inspection of the current result, deciding step-by-step which direction to take, and switching from one segmentation modality to another. Experiments with the segmentation and classification of multitemporal SAR images prove the potential of the interactive approach and of the TS-MRF tool.

We address the problem of SAR despeckling by resorting to nonlocal filtering guided by an optical image. In fact, given the increasing availability of remote-sensing optical images, it makes perfect sense trying to use them to improve the performance of despeckling. Our technique exploits the optical image to reliably estimate the statistical similarity among pixels, which is used to evaluate the weights of nonlocal filtering. Optical data are not used to estimate SAR values, but only to guide the overall process. In addition, they are discarded altogether in regions where SAR and optical images present different local geometries, identified by a preliminary classification step, avoiding thus any additional distortion. Experimental results show the proposed approach to provide images of better quality than state-of-the-art conventional filters.

The segmentation of very high resolution (VHR) images portraying complex urban scenarios is a rather challenging problem. In particular, great attention must be devoted to preserve fine man-made details, of major interest for most user applications. For this reason, edge-based segmentation methods are likely preferable to region-based methods. The latter, in fact, e.g. [1], [2], succeed in taking into account long range interactions and hence perform typically well in terms of “global” accuracy, but exhibit a lower “local” accuracy with respect to former, [3].

The Parallel Proximal Algorithm (PPXA+) has been recently introduced as an efficient tool for solving convex optimization problems. It has proved particularly effective in the context of stereo vision, used as the methodological core of a novel disparity estimation technique. In this work, the main methodological issues limiting the efficient parallelization of this technique are addressed, and further modifications are proposed to enable and optimize the design of parallel implementations. Finally, actual implementations that fit both the multi-core CPU and GPU devices are provided and tested to validate the performance potential of the proposed technique.

With the raise of cloud computing infrastructures on one side and the increased accessibility of parallel computational devices on the other, such as GPUs and multi-core CPUs, parallel programming has recently gained a renewed interest. This is particularly true in the domain of video coding, where the complexity and time consumption of the algorithms tend to limit the access to the core technology. In this work, we focus on the motion estimation problem, well-known to be the most time consuming step of a majority of video coding techniques. By relying on the use of the OpenCL standard, which provides a cross-platform framework for parallel programming, we propose here a scalable CPU/GPU implementation of the full search motion estimation algorithm (FSBM), and study its performances also with respect to the issues raised by the use of OpenCL.

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et a ̀ la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Information mining systems typically do not carry out image segmentation because a single algorithm could never perform well on the wide variety of sources and user applications encountered in practice. On the other hand, a large number of tools have been proposed in the literature that handle specific segmentation tasks very well. Dynamic segmentation is a possible solution, where the image is split recursively, in a hierarchical fashion, and different tools are used at each step to address specific segmentation tasks. In this work, the segmentation of a high-resolution test image is used as a running example and as a proof of concept of the potential of this approach.

The Texture Fragmentation and Reconstruction (TFR) algorithm, recently proposed for the segmentation of textured images, has been applied with promising results to high-resolution remote-sensing images. The algorithm provides a sequence of nested segmentation maps which allow the analysis at various scales of observation. However, the performance which is very good at large scales, with complex semantic areas retrieved with remarkable accuracy, becomes less satisfactory at finer scales. In this paper we propose to use the TFR in a recursive fashion, segmenting the image in just two regions, initially, with each region further segmented only if relevant subregions emerge. The recursive TFR allows one to better adapt to local statistics and to extract significant textures also at finer scales. Early experimental results validate the effectiveness of the new algorithm.

The Texture Fragmentation and Reconstruction (TFR) algorithm, recently proposed for the segmentation of textured images, has been applied with promising results to high-resolution remote-sensing images. The algorithm provides a sequence of nested segmentation maps which allow the analysis at various scales of observation. Although for most test images TFR has proven able to recognize major semantic areas, some failures have also been observed due to the presence of large background regions that span the whole image and prevent the formation of distinct local textures. In this paper we introduce a new step in the TFR processing flow which detects background regions and divides them in multiple homogeneous fragments based on their geometric level properties. To this end, connected regions are first reduced to atomic components through a watershed-like transform, and then clustered again based on the features of the associated region-adjacency graph. Early experimental results prove the effectiveness of the new processing step, and its beneficial effect on the whole algorithm.

Segmentation of very high resolution remote-sensing images cannot rely only on spectral information, quite limited here for technological reasons, but must take into account also the rich textural information available. To this end, we proposed recently the Texture Fragmentation and Reconstruction (TFR) algorithm, based on a split-and-merge paradigm, which provides a sequence of nested segmentation maps, at various scales of observation. Early experiments on several high-resolution test images confirm the potential of TFR, but there is room for further improvements under various points of view. In this paper we describe the TFR algorithm and, starting from the analysis of some critical results propose two new version that address and solve some of its weak points.

In this paper, we propose a new algorithm for the segmentation of multiresolution remote-sensing images, which fits into the general split-and-merge paradigm. The splitting phase singles out clusters of connected regions that share the same spatial and spectral characteristics. These clusters are then regarded as atomic elements of more complex structures, particularly textures, that are gradually retrieved during the merging phase. The whole process is based on a recently developed hierarchical model of the image, which accurately describes its textural properties. In order to reduce the computational burden and preserve contours at the highest spatial definition, the algorithm works on the high-resolution panchromatic data first, using low-resolution full spectral information only at a later stage to refine the segmentation. It is completely unsupervised, with just a few parameters set at the beginning, and its final product is not a single segmentation map but rather a sequence of nested maps which provide a hierarchical description of the image, at various scales of observations. The first experimental results, obtained on a remote-sensing Ikonos image, are very encouraging and confirm the algorithm potential.

In this paper, we present a novel multiscale texture model and a related algorithm for the unsupervised segmentation of color images. Elementary textures are characterized by their spatial interactions with neighboring regions along selected directions. Such interactions are modeled, in turn, by means of a set of Markov chains, one for each direction, whose parameters are collected in a feature vector that synthetically describes the texture. Based on the feature vectors, the texture are then recursively merged, giving rise to larger and more complex textures, which appear at different scales of observation: accordingly, the model is named Hierarchical Multiple Markov Chain (H-MMC). The Texture Fragmentation and Reconstruction (TFR) algorithm, addresses the unsupervised segmentation problem based on the H-MMC model. The "fragmentation" step allows one to find the elementary textures of the model, while the "reconstruction" step defines the hierarchical image segmentation based on a probabilistic measure (texture score) which takes into account both region scale and inter-region interactions. The performance of the proposed method was assessed through the Prague segmentation benchmark, based on mosaics of real natural textures, and also tested on real-world natural and remote sensing images.

Multispectral and synthetic aperture radar (SAR) images are known to exhibit complementary properties: unlike optical sensors, SAR provides information about the soil roughness and moisture, and acquires useful data despite clouds and Sun-illumination conditions. However, the analysis of the resulting images turns out to be more difficult, as compared to the use of optical imagery, due to the noise-like speckle phenomenon. In order to exploit this complementarity for classification purposes, a criticality relies in the definition of accurate joint optical-SAR statistical models, due to the different physical natures of these two data typologies and to the corresponding differences in the related parametric models. In this paper, a region-based semiparametric classification technique is proposed for multisensor optical-SAR images. The method combines the tree-structured Markov random field approach to segmentation with the dependence tree approach to probability density estimation and with case-specific bivariate models for the distributions of optical and SAR data. A Bayesian decision rule is formulated at the segment level in order to incorporate spatial-contextual information and to gain robustness against noise.

We propose here a new algorithm for the unsupervised segmentation of multiresolution remote-sensing images. After a first segmentation step on the high-resolution panchromatic data, the image is converted in a set of disjoint regions, which are then clustered and merged progressively, based on multispectral, spatial and textural properties, producing a sequence of nested segmentation maps which provide a thorough and multi-scale description of the image. The algorithm is fast, since it works mainly at a region level, and preserves fine details thanks to the initial step at the high-resolution level. Experimental results on IKONOS data confirm the algorithm potential and point out to a few problems to address in future research.

The region-based description of multispectral images enables important high-level tasks such as data mining and retrieval, and region-of-interest selection. In order to obtain an efficient representation of such images we resort to adaptive transform coding techniques. Such techniques, however, require a considerable information overhead, which must be carefully managed to obtain a satisfactory rate-distortion performance. In this work we develop several region-based coding schemes and compare them with conventional (non-adaptive) and class-based schemes, so as to single out the rate-distortion gains/losses of this approach.

Remote-sensing images are often composed by a hierarchy of nested regions, with complex regions that are regarded as homogeneous at some observation scale, but can be further segmented at finer scales. Tree-structured Markov random fields (TS-MRF) allow one to model such images, and to develop efficient segmentation algorithms for them. TS-MRF are traditionally based on binary trees of classes, but the use of generic trees, with more degrees of freedom, can likely provide a better performance, as was shown with reference to synthetic images. Here we build upon the ideas proposed to devise a segmentation algorithm that works effectively, and with a limited computational burden, on real-world remote sensing images.