Semi-supervised domain adaptation aims to transfer knowledge from a labeled source domain to a scarcely labeled target domain, despite distribution shifts. The challenge becomes greater when source and target data differ in acquisition modality, as in remote sensing where variations in sensor type (e.g., optical vs. radar), spectral properties (e.g., RGB vs. multispectral), or spatial resolution are common. This challenging scenario, known as Semi-Supervised Heterogeneous Domain Adaptation (SSHDA), requires learning across modalities with limited target labels. In this work, we propose SAHARA (Semi-supervised Adaptation in Heterogeneous domains via conditional Adversarial Representation disentanglement and Adaptive pseudo-labeling), a new method for SSHDA that combines conditional adversarial feature adaptation with dynamic pseudo-labeling to learn domain-invariant features and handle extremely scarce target annotations. Experiments on two heterogeneous remote sensing benchmarks for scene classification, conducted with both convolutional and transformer-based backbones, demonstrate that SAHARA consistently outperforms existing SSHDA and semi-supervised methods. The code is available at https: //TO-BE-DISCLOSED-UPON-ACCEPTANCE.

Hyperspectral pansharpening has received much attention in recent years due to technological and methodological advances that open the door to new application scenarios. However, research on this topic is only now gaining momentum. The most popular methods are still borrowed from the more mature field of multispectral pansharpening and often overlook the unique challenges posed by hyperspectral data fusion, such asi)the very large number of bands,ii)the overwhelming noise in selected spectral ranges,iii)the significant spectral mismatch between panchromatic and hyperspectral components,iv)a typically high resolution ratio. Imprecise data modeling especially affects spectral fidelity. Even state-of-the-art methods perform well in certain spectral ranges and much worse in others, failing to ensure consistent quality across all bands, with the risk of generating unreliable results. Here, we propose a hyperspectral pansharpening method that explicitly addresses this problem and ensures uniform spectral quality. To this end, a single lightweight neural network is used, with weights that adapt on the fly to each band. During fine-tuning, the spatial loss is turned on and off to ensure a fast convergence of the spectral loss to the desired level, according to a hysteresis-like dynamic. Furthermore, the spatial loss itself is appropriately redefined to account for nonlinear dependencies between panchromatic and spectral bands. Overall, the proposed method is fully unsupervised, with no prior training on external data, flexible, and low-complexity. Experiments on a recently published benchmarking toolbox show that it ensures excellent sharpening quality, competitive with the state-of-the-art, consistently across all bands. The software code and the full set of results are shared online on https://github.com/giu-guarino/rho-PNN.

Monitoring particulate matter (PM) is of critical importance due to its significant impact on human health. Ground stations provide highly accurate measurements of various pollutants on a local scale. However, the limited distribution of these stations makes achieving global coverage challenging. To address this limitation, satellite imagery serves as a valuable resource, offering wide-area PM estimates in near real-time through abundant data and frequent revisit intervals. In contrast to other studies, this work introduces deep learning (DL) models to estimate ground-level PM concentration maps over Europe. These models rely exclusively on radiance data from the Sentinel-5P satellite, forgoing auxiliary information, such as meteorological data, which are commonly incorporated in similar studies. The proposed approach has demonstrated both robust estimation accuracy and effective generalization capabilities. Furthermore, the estimated PM concentration maps have been validated against ground-based measurements, showing superior performance with respect to widely used models and datasets that consider meteorological inputs. The dataset and the code are available here:https://github.com/antoniomazza88/PMUnet.

In the last years it has been observed a growing interest toward deep leaning techniques for the pansharpening of multiresolution images. Due to the lack of data with ground truth, most deep learning solutions exploit synthetic reduced-resolution data to carry out supervised training. Such an approach, though granting an easy way to step over the lack of labeled data, has shown its limitations due to the statistical mismatch between real full-resolution data and synthetic reduced-resolution data, which eventually impacts on the generalization capacity of the trained models. This has motivated a recent paradigm shift from supervised to unsupervised learning frameworks for pansharpening. Unsupervised schemes, however, involve the definition of more sophisticated loss functions which comprise, at least, two fundamental terms: one responsible for the spectral quality, meant as consistency between the pansharpened image and the input multispectral component; the other accounting for the spatial quality, read as consistency between the output and the panchromatic input. Despite the very good results shown by many such unsupervised solutions, a minor attention has been devoted to the investigation of the interaction between the above mentioned loss terms and to their proper balance to grant stability while pursuing accuracy. This work aims to explore to what extent unsupervised spatial and spectral consistency losses can be reliably combined without impairing quality.

In this paper, a representative set of state-of-the-art methods for hyperspectral pansharpening, comprising both model- and deep learning-based ones, are reviewed and compared on four datasets from the PRISMA mission. The experimental analysis has been carried out using the most credited pansharpening quality indexes, complemented by a subjective visual inspection of sample results. The obtained outcomes have provided us a preview of the strengths and weaknesses of the latest solutions to the problem at hand, paving the way for future research lines from both the methodological and quality assessment perspectives.

This work proposes a hybrid approach to address the pansharpening of hyperspectral images, which mixes the use of a recently proposed CNN-based solution and a classical solution such as the Gram Schmidt Adaptive method (GSA). The hyperspectral datacube is split in two sets of bands, those falling in the visible range and the remaining ones. The first set is pansharpened using the GSA approach which has proven to grant very high quality results in this range. The remaining bands, whose relationship with the panchromatic band is much weaker, undergo a fusion process based on a recently proposed hyperspectral pansharpening method known as Rolling hyperspectral Pansharpening Neural Network (R-PNN). By doing so, we are able to take the best features from both solutions, getting higher quality results compared to the marginal use of any of the two methods.

This work deals with the estimation of the tropospheric vertical column density of nitrogen dioxide from Sentinel-5P radiance data using convolutional neural networks. The current processing chain to retrieve this information from Sentinel-5P data requires a complex, computationally demanding, physical modeling that involves the use of additional side information such as meteorological variables, which are not always available. Therefore, in this proof-of-concept study, we explored the feasibility of an estimation exclusively using radiance data from Sentinel-5P, leveraging on the powerful representational capacity of deep neural networks. Preliminary results are very promising encouraging further investigation.

Hyperspectral pansharpening is receiving a growing interest since the last few years as testified by a large number of research papers and challenges. It consists in a pixel-level fusion between a lower-resolution hyperspectral datacube and a higher-resolution single-band image, the panchromatic image, with the goal of providing a hyperspectral datacube at panchromatic resolution. Thanks to their powerful representational capabilities, deep learning models have succeeded to provide unprecedented results on many general purpose image processing tasks. However, when moving to domain specific problems, as in this case, the advantages with respect to traditional model-based approaches are much lesser clear-cut due to several contextual reasons. Scarcity of training data, lack of ground-truth, data shape variability, are some such factors that limit the generalization capacity of the state-of-the-art deep learning networks for hyperspectral pansharpening. To cope with these limitations, in this work we propose a new deep learning method which inherits a simple single-band unsupervised pansharpening model nested in a sequential band-wise adaptive scheme, where each band is pansharpened refining the model tuned on the preceding one. By doing so, a simple model is propagated along the wavelength dimension, adaptively and flexibly, with no need to have a fixed number of spectral bands, and, with no need to dispose of large, expensive and labeled training datasets. The proposed method achieves very good results on our datasets, outperforming both traditional and deep learning reference methods. The implementation of the proposed method can be found on https://github.com/giu-guarino/R-PNN.

The recent paradigm shift from model-based to data-driven approaches has involved a growing number of data-fusion tasks. Specifically for pansharpening, unsupervised deep learning methods have been recently explored with the goal of overcoming the generalization limits shown by early pansharpening convolutional neural networks based on supervised training schemes. Furthermore, some of these exploit the target-adaptive modality to face the scarcity of training data. On the downside, combining usupervised training and target adaptivity causes a non-negligible increase of the computational cost. This work presents a new target adaptive scheme that allows to keep limited the computational cost at inference time while preserving accuracy.

This work proposes a simple yet effective method to adapt unsupervised convolutional neural networks for pansharpening of multispectral images to the problem of hyperspectral image pansharpening, i.e., the fusion of a single high-resolution panchromatic band with a low-resolution hyperspectral data cube. This is achieved by means of a PCA transformation which allows to compact the most of the HS image energy in a few bands, which are then suitably super-resolved using a pansharpening network designed for few spectral bands. Our experiments show very encouraging results which compare favorably against the state-of-the-art methods.

Optical remote sensing images are subject to cloud phenomena that can cause information loss in Earth observation. The main alternative is represented by the synthetic aperture radar images. However, many Earth monitoring applications exploit specific spectral features defined for multispectral data only. In this work, we propose a method that aims to recover several spectral features through deep learning-based data fusion of Sentinel-1 and Sentinel-2 time-series. The proposed approach has been experimentally validated for radiometric indexes such as the normalized difference vegetation index, the normalized difference water index, the soil-adjusted vegetation index and the atmospherically resistant vegetation index. Both numerical and visual results show that the proposed solution outperforms consistently the compared methods.

This work proposes a simple yet effective method to adapt unsupervised convolutional neural networks from multispectral to hyperspectral pansharpening. Thus, it focuses on the fusion of a single high-resolution panchromatic band with a low-resolution hyperspectral data cube. This is achieved by means of a decorrelation transform, following the principal component analysis approach, which enables the compression of a significant portion of the hyperspectral image energy into a few bands. Afterwards, a suitably adapted pansharpening network designed for four spectral bands is used to super-resolve only the principal components. Experiments demonstrate high performance in both quantitative and qualitative evaluations, favorably comparing against state-of-the-art methods.

In latest years, deep learning has gained a leading role in the pansharpening of multiresolution images. Given the lack of ground truth data, most deep learning-based methods carry out supervised training in a reduced-resolution domain. However, models trained on downsized images tend to perform poorly on high-resolution target images. For this reason, several research groups are now turning to unsupervised training in the full-resolution domain, through the definition of appropriate loss functions and training paradigms. In this context, we have recently proposed a full-resolution training framework which can be applied to many existing architectures. Here, we propose a new deep learning-based pansharpening model that fully exploits the potential of this approach and provides cutting-edge performance. Besides architectural improvements with respect to previous work, such as the use of residual attention modules, the proposed model features a novel loss function that jointly promotes the spectral and spatial quality of the pansharpened data. In addition, thanks to a new fine-tuning strategy, it improves inference-time adaptation to target images. Experiments on a large variety of test images, performed in challenging scenarios, demonstrate that the proposed method compares favorably with the state of the art both in terms of numerical results and visual output. Code is available online at https://github.com/matciotola/Lambda-PNN.

In this work is presented a new adversarial training framework for deep learning neural networks for super-resolution of Sentinel 2 images, exploiting the data fusion techniques on 10 and 20 meters bands. The proposed scheme is fully convolutional and tries to answer the need for generalization in scale, producing realistic and detailed accurate images. Furthermore, the presence of a$\mathcal{L}_{1}$loss limits the instability of GAN training, limiting possible problems of spectral dis-tortion. In our preliminary experiments, the GAN training scheme has shown comparable results in comparison with the baseline approach.

Phase unwrapping (PU) is among the most critical tasks in synthetic aperture radar (SAR) interferometry (InSAR). Due to the presence of noise, the interferogram usually presents phase inconsistencies, also called residues, which imply a nonunivocal solution. This work investigates the PU problem from a semantic segmentation perspective by exploiting convolutional neural network (CNN) models. In particular, by exploiting a popular deep-learning architecture, we introduce the interferometric coherence as an input feature and analyze the performance increase against classical methods. For the network training, we generate a variegated data set by introducing a controlled number of phase residues, and considering both synthetic and real InSAR data. Eventually, we compare the proposed method to state-of-the-art algorithms on synthetic and real InSAR data taken from the TanDEM-X mission, obtaining encouraging results.

In recent years, there has been a growing interest in deep learning-based pansharpening. Thus far, research has mainly focused on architectures. Nonetheless, model training is an equally important issue. A first problem is the absence of ground truths, unavoidable in pansharpening. This is often addressed by training networks in a reduced-resolution domain and using the original data as ground truth, relying on an implicit scale invariance assumption. However, on full-resolution images, results are often disappointing, suggesting such invariance not to hold. A further problem is the scarcity of training data, which causes a limited generalization ability and a poor performance on off-training-test images. In this article, we propose a full-resolution training framework for deep learning-based pansharpening. The framework is fully general and can be used for any deep learning-based pansharpening model. Training takes place in the high-resolution domain, relying only on the original data, thus avoiding any loss of information. To ensure spectral and spatial fidelity, a suitable two-component loss is defined. The spectral component enforces consistency between the pansharpened output and the low-resolution multispectral input. The spatial component, computed at high resolution, maximizes the local correlation between each pansharpened band and the panchromatic input. At testing time, the target-adaptive operating modality is adopted, achieving good generalization with a limited computational overhead. Experiments carried out on WorldView-3, WorldView-2, and GeoEye-1 images show that methods trained with the proposed framework guarantee a pretty good performance in terms of both full-resolution numerical indexes and visual quality.

This work presents a new unsupervised framework for training deep learning models for super-resolution of Sentinel-2 images by fusion of its 10-m and 20-m bands. The proposed scheme avoids the resolution downgrade process needed to generate training data in the supervised case. On the other hand, a proper loss that accounts for cycle-consistency between the network prediction and the input components to be fused is proposed. Despite its unsupervised nature, in our preliminary experiments the proposed scheme has shown promising results in comparison to the supervised approach. Besides, by construction of the proposed loss, the resulting trained network can be ascribed to the class of multi-resolution analysis methods.

Cloud segmentation of remotely sensed multispectral images is an important topic not only for weather forecast but, more in general, for establishing when the sensed data actually relate to the soil so that can be reliably used for some monitoring purpose. In this work, leveraging on the capability of convolutional neural networks to accurately approximate complex relationships between raw data and higher-level products, we propose a U-Net-like solution conceived for Sentinel-2 images. In order to face the scarsity of training data, a proper domain adaptation strategy has been pursued, which resorts to a labeled Landsat-8 dataset. Preliminary results show a consistent improvement over standard tools.

The rise of deep learning has impacted profoundly all aspects of image processing and remote sensing. Following this trend, in the last few years, a large number of data-driven methods have been proposed also for SAR image despeckling. However, in spite of this large effort, only limited performance gains have been observed. We believe this is mostly due to the use of training sets that are only partially fit to the task, and sometimes plain wrong. In this work we assess experimentally the impact of training set design on the performance of SAR image despeckling with the goal of highlighting solid guidelines for sensible training.

To cope with the lack of input-output training samples, deep learning (DL) methods for pansharpening usually resort to Wald's protocol or other similar downscaling processes. By doing so, the scaled versions of the multispectral (MS) and panchromatic (PAN) components serve as input while the original MS plays as output during the training phase. As a side effect, the informational gap between reduced and full scales causes a mismatch between the training and test phases. In fact, DL methods typically provide a pretty good performance at reduced scale, with a good margin over traditional solutions that tends to vanish in the full-resolution framework. In this work, we propose a training framework that involves both the reduced and the full scale versions of the multiresolution image samples. This is achieved thanks to a suitably defined loss which comprises costs for both scales. Our numerical and visual experimental results confirm that the proposed approach provides an improved performance in the full-resolution case.

Nonlocal methods are state-of-the-art in SAR despeckling, thanks to their ability to exploit image self-similarity. Given sufficient training data, however, methods based on deep learning have proven highly competitive. Therefore, to take the best of both approaches, we investigate the use of deep learning to improve nonlocal despeckling. We use plain non-iterative nonlocal means despeckling, with weights provided, for each estimation window, by a suitably trained deep CNN. Experiments on synthetic and real SAR data prove this approach to outperform conventional nonlocal methods.

We propose a new method for synthetic aperture radar (SAR) image despeckling, which leverages information drawn from coregistered optical imagery. Filtering is performed by patchwise nonlocal means, working exclusively on SAR data. However, the filtering weights are computed by taking into account also the optical guide, which is much cleaner than the SAR image, and hence more discriminative. To avoid injecting optical-domain information into the filtered image, an SAR-domain statistical test is preliminarily performed to reject right away any risky predictor. Experiments on two SAR-optical data sets prove the proposed method to suppress very effectively the speckle, preserving structural details, and without introducing significant filtering artifacts. Overall, the proposed method compares favorably with all the state-of-the-art despeckling filters, and also with our own previous optical-guided filter.

Sentinel-2 data represent a rich source of information for the community due to the free access and to the temporal-spatial coverage assured. However, some of the spectral bands are sensed at reduced resolution due to a compromise between technological limitations and Copernicus program's objectives. For this reason in this work we present a new super-resolution method based on Convolutional Neural Networks (CNNs) to rise the resolution of the short wave infra-red (SWIR) band from 20 to 10 meters, that is the highest resolution provided. This is accomplished by fusing the target band with the finer-resolution ones. The proposed solution compares favourably against several alternative methods according to different quality indexes. In addition we have also tested the use of the super-resolved band from an applicative perspective by detecting water basins through the Modified Normalized Difference Water Index (MNDWI).

In this paper, the problem of the spatial-spectral classification of very high-resolution optical images is addressed using a kernel- and region-based approach. A novel method based on integrating region-based or object-based information into a kernel machine is developed. A Gaussian process model is used to characterize each segment in a segmentation map and to define a region-based admissible kernel accordingly. This kernel is combined with a marker-controlled watershed segmentation that incorporates scale adaptivity. Spatial-spectral fusion capabilities are also ensured by combining the resulting classification method with composite kernels.

Since optical remote sensing images are useless in cloudy conditions, a possible alternative is to resort to synthetic aperture radar (SAR) images. However, many conventional techniques for Earth monitoring applications require specific spectral features which are defined only for multispectral data. For this reason, in this work we propose to estimate missing spectral features through data fusion and deep learning, exploiting both temporal and cross-sensor dependencies on Sentinel-1 and Sentinel-2 time-series. The proposed approach, validated focusing on the estimation of the normalized difference vegetation index (NDVI), shows very interesting results with a large performance gain over the linear regression approach according to several accuracy indicators.

Fusing a multispectral image with a co-registered higher resolution single panchromatic band, provided by any multiresolution satellite systems, to rise the resolution of the former to that of the latter is known as pansharpening, and can be regarded as a guided super-resolution problem. Recently the use of convolutional neural networks (CNNs) has been extended to the pansharpening problem achieving state-of-the-art performance. Following this research line, the objective of this work was two-fold: provide a trained CNN model fitted to a specific sensor (WorldView-3) and explore a range of architectural configurations varied in both width and depth, seeking for the optimal one. Numerical and visual results show that the proposed solution compares favourably against reference methods.

We recently proposed a convolutional neural network (CNN) for remote sensing image pansharpening obtaining a significant performance gain over the state of the art. In this paper, we explore a number of architectural and training variations to this baseline, achieving further performance gains with a lightweight network that trains very fast. Leveraging on this latter property, we propose a target-adaptive usage modality that ensures a very good performance also in the presence of a mismatch with respect to the training set and even across different sensors. The proposed method, published online as an off-the-shelf software tool, allows users to perform fast and high-quality CNN-based pansharpening of their own target images on general-purpose hardware.

We propose a despeckling algorithm for multitemporal synthetic aperture radar (SAR) images based on the concepts of block-matching and collaborative filtering. It relies on the nonlocal approach, and it is the extension of SAR-BM3D for dealing with multitemporal data. The technique comprises two passes, each one performing grouping, collaborative filtering, and aggregation. In particular, the first pass performs both the spatial and temporal filtering, while the second pass only the spatial one. To avoid increasing the computational cost of the technique, we resort to lookup tables for the distance computation in the block-matching phases. The experiments show that the proposed algorithm compares favorably with respect to state-of-the-art reference techniques, with better results both on simulated speckled images and on real multitemporal SAR images.

We propose a new approach to SAR despeckling, based on the combination of multiple alternative estimates of the same data. The many despeckling methods proposed in the literature possess different and often complementary strengths and weaknesses. Given a reliable pixel-wise classification of the image, one can take advantage of this diversity by selecting the more appropriate combination of estimators for each image region. We implement a simplified version of this approach, using soft classification and two state-of-the-art despeckling tools, with opposite properties, as basic estimators. Experiments on real-world high-resolution SAR images prove the effectiveness of the proposed technique and confirm the potential of the whole approach.

In this paper a new object-oriented segmentation method for high-resolution remote sensing images is proposed. To limit computational complexity, a preliminary superpixel representation of the image is obtained by means of a suitable watershed transform. Then, a region adjacency graph is associated with the superpixels, with edge weights accounting for region similarity/dissimilarity. The final segmentation is then obtained by means of a graph-cutting approach, following a correlation clustering formulation. The optimal cut can be obtained by solving a Integer Linear Programming (ILP) problem, whose complexity, however, grows rapidly with the image size. Much faster near-optimal solutions are obtained, here, with a greedy solution. Experiments on a real-world high-resolution remote sensing image prove the potential of the approach.

We propose an interactive tool for designing ground-truth maps associated with multi-resolution remote sensing images. The target image is first segmented at object level by means of an edge-preserving algorithm. Then, a pre-classification defines groups of segments that are homogeneous both in spectral response and size. Finally, suitable candidate segments are selected and shown to the supervisor for inspection and labeling or possible rejection, in an iterative process, until the desired image covering is reached. Experimental results show that the proposed solution allows one to easily and quickly obtain ground-truth maps which are both locally and globally accurate, and where all classes are represented in a balanced manner.

A new technique for the segmentation of single- and multiresolution (MR) remote sensing images is proposed. To guarantee the preservation of details at fine scales, edge-based watershed is used, with automatically generated markers that help in limiting oversegmentation. For MR images, the panchromatic and multispectral components are processed independently, extracting both the edge maps and the morphological and spectral markers that are eventually fused at the highest resolution, thus avoiding any information loss induced by pansharpening. Numerical results on object layer extraction and simple classification tasks prove the proposed techniques to provide accurate segmentation maps, which preserve fine details and, contrary to state-of-the-art products, can single out objects equally well at very different scales.

Reliable segmentation of SAR images requires some forms of user supervision: we resort here to the interactive version of the Tree-Structured Markov Random Field (TS-MRF) segmentation suite. The TS-MRF model, and the associated segmentation tool, provide a flexible and spatially adaptive description of the data. In the interactive version, the user can drive the process based on the inspection of the current result, deciding step-by-step which direction to take, and switching from one segmentation modality to another. Experiments with the segmentation and classification of multitemporal SAR images prove the potential of the interactive approach and of the TS-MRF tool.

Despeckling techniques based on the nonlocal approach provide an excellent performance, but exhibit also a remarkable complexity, unsuited to time-critical applications. In this letter, we propose a fast nonlocal despeckling filter. Starting from the recent SAR-BM3D algorithm, we propose to use a variable-size search area driven by the activity level of each patch, and a probabilistic early termination approach that exploits speckle statistics in order to speed up block matching. Finally, the use of look-up tables helps in further reducing the processing costs. The technique proposed conjugates excellent performance and low complexity, as demonstrated on both simulated and real-world SAR images and on a dedicated SAR despeckling benchmark.

The segmentation of very high resolution (VHR) images portraying complex urban scenarios is a rather challenging problem. In particular, great attention must be devoted to preserve fine man-made details, of major interest for most user applications. For this reason, edge-based segmentation methods are likely preferable to region-based methods. The latter, in fact, e.g. [1], [2], succeed in taking into account long range interactions and hence perform typically well in terms of “global” accuracy, but exhibit a lower “local” accuracy with respect to former, [3].

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et a ̀ la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Information mining systems typically do not carry out image segmentation because a single algorithm could never perform well on the wide variety of sources and user applications encountered in practice. On the other hand, a large number of tools have been proposed in the literature that handle specific segmentation tasks very well. Dynamic segmentation is a possible solution, where the image is split recursively, in a hierarchical fashion, and different tools are used at each step to address specific segmentation tasks. In this work, the segmentation of a high-resolution test image is used as a running example and as a proof of concept of the potential of this approach.

Speckle reduction is a key step in several SAR image processing procedures. In this paper, a new despeckling technique based on the “nonlocal” denoising filter BM3D [1] is presented. The filter has been modified in order to take into account SAR image characteristics. The experimental results, conducted on both synthetic and real SAR images, confirm the potential of the proposed approach.

The Texture Fragmentation and Reconstruction (TFR) algorithm, recently proposed for the segmentation of textured images, has been applied with promising results to high-resolution remote-sensing images. The algorithm provides a sequence of nested segmentation maps which allow the analysis at various scales of observation. However, the performance which is very good at large scales, with complex semantic areas retrieved with remarkable accuracy, becomes less satisfactory at finer scales. In this paper we propose to use the TFR in a recursive fashion, segmenting the image in just two regions, initially, with each region further segmented only if relevant subregions emerge. The recursive TFR allows one to better adapt to local statistics and to extract significant textures also at finer scales. Early experimental results validate the effectiveness of the new algorithm.

The Texture Fragmentation and Reconstruction (TFR) algorithm, recently proposed for the segmentation of textured images, has been applied with promising results to high-resolution remote-sensing images. The algorithm provides a sequence of nested segmentation maps which allow the analysis at various scales of observation. Although for most test images TFR has proven able to recognize major semantic areas, some failures have also been observed due to the presence of large background regions that span the whole image and prevent the formation of distinct local textures. In this paper we introduce a new step in the TFR processing flow which detects background regions and divides them in multiple homogeneous fragments based on their geometric level properties. To this end, connected regions are first reduced to atomic components through a watershed-like transform, and then clustered again based on the features of the associated region-adjacency graph. Early experimental results prove the effectiveness of the new processing step, and its beneficial effect on the whole algorithm.

Segmentation of very high resolution remote-sensing images cannot rely only on spectral information, quite limited here for technological reasons, but must take into account also the rich textural information available. To this end, we proposed recently the Texture Fragmentation and Reconstruction (TFR) algorithm, based on a split-and-merge paradigm, which provides a sequence of nested segmentation maps, at various scales of observation. Early experiments on several high-resolution test images confirm the potential of TFR, but there is room for further improvements under various points of view. In this paper we describe the TFR algorithm and, starting from the analysis of some critical results propose two new version that address and solve some of its weak points.

In this paper, we propose a new algorithm for the segmentation of multiresolution remote-sensing images, which fits into the general split-and-merge paradigm. The splitting phase singles out clusters of connected regions that share the same spatial and spectral characteristics. These clusters are then regarded as atomic elements of more complex structures, particularly textures, that are gradually retrieved during the merging phase. The whole process is based on a recently developed hierarchical model of the image, which accurately describes its textural properties. In order to reduce the computational burden and preserve contours at the highest spatial definition, the algorithm works on the high-resolution panchromatic data first, using low-resolution full spectral information only at a later stage to refine the segmentation. It is completely unsupervised, with just a few parameters set at the beginning, and its final product is not a single segmentation map but rather a sequence of nested maps which provide a hierarchical description of the image, at various scales of observations. The first experimental results, obtained on a remote-sensing Ikonos image, are very encouraging and confirm the algorithm potential.

In this paper, we present a novel multiscale texture model and a related algorithm for the unsupervised segmentation of color images. Elementary textures are characterized by their spatial interactions with neighboring regions along selected directions. Such interactions are modeled, in turn, by means of a set of Markov chains, one for each direction, whose parameters are collected in a feature vector that synthetically describes the texture. Based on the feature vectors, the texture are then recursively merged, giving rise to larger and more complex textures, which appear at different scales of observation: accordingly, the model is named Hierarchical Multiple Markov Chain (H-MMC). The Texture Fragmentation and Reconstruction (TFR) algorithm, addresses the unsupervised segmentation problem based on the H-MMC model. The "fragmentation" step allows one to find the elementary textures of the model, while the "reconstruction" step defines the hierarchical image segmentation based on a probabilistic measure (texture score) which takes into account both region scale and inter-region interactions. The performance of the proposed method was assessed through the Prague segmentation benchmark, based on mosaics of real natural textures, and also tested on real-world natural and remote sensing images.

Multispectral and synthetic aperture radar (SAR) images are known to exhibit complementary properties: unlike optical sensors, SAR provides information about the soil roughness and moisture, and acquires useful data despite clouds and Sun-illumination conditions. However, the analysis of the resulting images turns out to be more difficult, as compared to the use of optical imagery, due to the noise-like speckle phenomenon. In order to exploit this complementarity for classification purposes, a criticality relies in the definition of accurate joint optical-SAR statistical models, due to the different physical natures of these two data typologies and to the corresponding differences in the related parametric models. In this paper, a region-based semiparametric classification technique is proposed for multisensor optical-SAR images. The method combines the tree-structured Markov random field approach to segmentation with the dependence tree approach to probability density estimation and with case-specific bivariate models for the distributions of optical and SAR data. A Bayesian decision rule is formulated at the segment level in order to incorporate spatial-contextual information and to gain robustness against noise.

A novel model for unsupervised segmentation of texture images is presented. The image to be segmented is first discretized and then a hierarchical finite-state region-based model is automatically coupled with the data by means of a sequential optimization scheme, namely the texture fragmentation and reconstruction (TFR) algorithm. Both intra- and inter-texture interactions are modeled, by means of an underlying hierarchical finite-state model, and eventually the segmentation task is addressed in a completely unsupervised manner. The output is then a nested segmentation, so that the user may decide the scale at which the segmentation has to be provided. TFR is composed of two steps: the former focuses on the estimation of the states at the finest level of the hierarchy, and is associated with an image fragmentation, or over-segmentation; the latter deals with the reconstruction of the hierarchy representing the textural interaction at different scales.

We propose here a new algorithm for the unsupervised segmentation of multiresolution remote-sensing images. After a first segmentation step on the high-resolution panchromatic data, the image is converted in a set of disjoint regions, which are then clustered and merged progressively, based on multispectral, spatial and textural properties, producing a sequence of nested segmentation maps which provide a thorough and multi-scale description of the image. The algorithm is fast, since it works mainly at a region level, and preserves fine details thanks to the initial step at the high-resolution level. Experimental results on IKONOS data confirm the algorithm potential and point out to a few problems to address in future research.

Remote-sensing images are often composed by a hierarchy of nested regions, with complex regions that are regarded as homogeneous at some observation scale, but can be further segmented at finer scales. Tree-structured Markov random fields (TS-MRF) allow one to model such images, and to develop efficient segmentation algorithms for them. TS-MRF are traditionally based on binary trees of classes, but the use of generic trees, with more degrees of freedom, can likely provide a better performance, as was shown with reference to synthetic images. Here we build upon the ideas proposed to devise a segmentation algorithm that works effectively, and with a limited computational burden, on real-world remote sensing images.

Most remote sensing images exhibit a clear hierarchical structure which can be taken into account by defining a suitable model for the unknown segmentation map. To this end, one can resort to the tree-structured Markov random field (MRF) model, which describes a K-ary field by means of a sequence of binary MRFs, each one corresponding to a node in the tree. Here we propose to use the tree-structured MRF model for supervised segmentation. The prior knowledge on the number of classes and their statistical features allows us to generalize the model so that the binary MRFs associated with the nodes can be adapted freely, together with their local parameters, to better fit the data. In addition, it allows us to define a suitable likelihood term to be coupled with the TS-MRF prior so as to obtain a precise global model of the image. Given the complete model, a recursive supervised segmentation algorithm is easily defined. Experiments on a test SPOT image prove the superior performance of the proposed algorithm with respect to other comparable MRF-based or variational algorithms.

Multitemporal remote sensing images are useful tools for many applications in natural resource management. Compression of this kind of data is an issue of interest, yet, only a few paper address it specifically, while general-purpose compression algorithms are not well suited to the problem, as they do not exploit the strong correlation among images of a multitemporal set of data. Here we propose a coding architecture for multitemporal images, which takes advantage of segmentation in order to compress data. Segmentation subdivides images into homogeneous regions, which can be efficiently and independently encoded. Moreover this architecture provides the user with a great flexibility in transmitting and retrieving only data of interest

We deal with the supervised segmentation of multi-temporal remote-sensing images following a statistical Bayesian approach. To take into account prior information on the class of images, like the correlation between neighboring pixels, as well as the available knowledge about the structure of the current image, we model the image as a tree-structured Markov random field. The data collected at two different dates are jointly processed as a single multi-component image, with the classes defined a priori based on ground truth information and grouped in changed and unchanged macro-classes. Experimental results in terms of classification accuracy prove the effectiveness of the proposed technique with respect to non-contextual methods, as well as to a disjoint approach. In addition, the classification tree allows for a direct interpretation of the result

We present a fast Bayesian algorithm for the segmentation of remote-sensing images. It alternates two processing steps, the binary Bayesian segmentation of regions, and the separation of non-connected same-class regions, which both present relatively low complexity. As a result, a detailed and reliable K-region segmentation map can be obtained in limited CPU-time. In addition, the map is organized in a tree-structure (not necessarily binary) which helps gaining insight about the meaning of component regions.

The Bayesian/MRF approach guarantees high-quality image segmentation, at the price of a significant computational cost. To limit complexity, one can resort to the recently proposed tree-structured MRF-based segmentation, which converts a Kclass problem in a sequence of much simpler binary tasks. The binary-tree structure imposed to the image, however, can also reduce segmentation accuracy. Here we propose an improved tree-structured segmentation algorithm, where disjoint regions, even belonging to the same class, are immediately split, giving rise to a generic tree structure, and are grouped again in classes only at the end of the algorithm. As a result, both segmentation speed and accuracy increase.

We present a new image segmentation algorithm based on a tree-structured binary MRF model. The image is recursively segmented in smaller and smaller regions until a stopping condition, local to each region, is met. Each elementary binary segmentation is obtained as the solution of a MAP estimation problem, with the region prior modeled as an MRF. Since only binary fields are used, and thanks to the tree structure, the algorithm is quite fast, and allows one to address the cluster validation problem in a seamless way. In addition, all field parameters are estimated locally, allowing for some spatial adaptivity. To improve segmentation accuracy, a split-and-merge procedure is also developed and a spatially adaptive MRF model is used. Numerical experiments on multispectral images show that the proposed algorithm is much faster than a similar reference algorithm based on "flat" MRF models, and its performance, in terms of segmentation accuracy and map smoothness, is comparable or even superior.