Core image processing tasks, such as super-resolution, denoising, deblurring, pansharpening, and atmospheric correction, underpin all optical remote sensing (RS) pipelines. Errors at this stage propagate through downstream applications, distorting land-cover maps, change detection, and climate records. Classical physics-based models capture sensor optics, radiometry, and geometry but struggle with complex noise and scene variability. In contrast, deep learning (DL) methods offer powerful data-driven solutions yet often act as closed boxes, ignoring physical constraints and overfitting to spurious patterns. Hybrid DL (HDL) approaches bridge this gap by integrating physical models with neural architectures, combining interpretability and data adaptivity. This article surveys the emerging landscape of HDL methods in RS image processing, outlining their theoretical foundations, motivations, and design philosophies. We categorize fusion strategies, from model-embedded schemes (e.g., plug-and-play (PnP) and unrolling) to model-guided learning (e.g., deep image prior (DIP) and unsupervised frameworks), and discuss how they enhance trust, robustness, and physical consistency in RS image analysis.

Optical satellite imagery is widely used for estimating ground movement in the aftermath of natural disasters such as earthquakes. This type of imagery enables detailed analysis of the factors and mechanisms that drive or influence these events. By using sub-pixel correlation algorithms, it provides precise displacement measurements (in the meter-to-centimeter range) and high spatial resolution (decimeter-to-centimeter level) by comparing images taken before and after the event. In this study, we present a deep neural network approach, trained on our new specific realistic dataset FaultDeform, to retrieve full-scale seismic ground motion displacement fields from optical satellite images with sub-pixel precision. The FaultDeform dataset, available at https://doi.org/10.57745/G02ZXZ, is the first satellite synthetic dataset tailored for ground motion estimation. We introduce the GeoFlowNet pipeline, utilizing a U-net architecture to solve the displacement estimation problem, delivering high-speed performance through GPU implementation, and outperforming current correlators in speed and precision. Comprehensive comparisons with state-of-the-art methods such as COSI-Corr, MicMac and CNN-DIS, and validation on real-world data from the 2019 Ridgecrest and 2013 Balochistan earthquakes showcases the robustness of our method. Codes are freely available: gricad-gitlab.univ-grenoble-alpes.fr/montagtr/GeoFlowNet.

Multispectral image (MS) and panchromatic image (PAN) fusion, which is also named as multispectral pansharpening, aims to obtain MS with high spatial resolution and high spectral resolution. However, due to the usual neglect of noise and blur generated in the imaging and transmission phases of data during training, many deep learning (DL) pansharpening methods fail to perform on the dataset containing noise and blur. To tackle this problem, a variational optimization-guided two-stage network (VOGTNet) for multispectral pansharpening is proposed in this work, and the performance of variational optimization (VO)-based pansharpening methods relies on prior information and estimates of spatial-spectral degradation from the target image to other two original images. Concretely, we propose a dual-branch fusion network (DBFN) based on supervised learning and train it by using the datasets containing noise and blur to generate the prior fusion result as the prior information that can remove noise and blur in the initial stage. Subsequently, we exploit the estimated spectral response function (SRF) and point spread function (PSF) to simulate the process of spatial-spectral degradation, respectively, thereby making the prior fusion result and the adaptive recovery model (ARM) jointly perform unsupervised learning on the original dataset to restore more image details and results in the generation of the high-resolution MSs in the second stage. Experimental results indicate that the proposed VOGTNet improves pansharpening performance and shows strong robustness against noise and blur. Furthermore, the proposed VOGTNet can be extended to be a general pansharpening framework, which can improve the ability to resist noise and blur of other supervised learning-based pansharpening methods. The source code is available at https://github.com/HZC-1998/VOGTNet.

In the last decade, novel hyperspectral cameras have been developed with particularly desirable characteristics of compactness and short acquisition time, retaining their potential to obtain spectral/spatial resolution competitive with respect to traditional cameras. However, a computational effort is required to recover an interpretable data cube. In this work we focus our attention on imaging spectrometers based on interferometry, for which the raw acquisition is an image whose spectral component is expressed as an interferogram. Previous works have focused on the inversion of such acquisition on a pixel-by-pixel basis within a Bayesian framework, leaving behind critical information on the spatial structure of the image data cube. In this work, we address this problem by integrating a spatial regularization for image reconstruction, showing that the combination of spectral and spatial regularizers leads to enhanced performances with respect to the pixelwise case. We compare our results with Plug-and-Play techniques, as its strategy to inject a set of denoisers from the literature can be implemented seamlessly with our physics-based formulation of the optimization problem.

This paper proposes an image translation method based on multi-scale fusion GAN (MFS) network. In MFS network, there are two modules: optical image generation sub-network (OGS), optical image generation sub-network (OGS) and SAR image regressive sub-network (SRS). Firstly, we design a multi-scale fusion generator (MFG) to perform SAR-to-optical image translation and SAR image regression in OGS and SRS. MFG can extract multi-scale features at different scales, and fuse shallow and deep features, which enriches the semantic information of features. Combined with the PatchGAN discriminator, SAR-to-Optical image translation can be effectively realized. Finally, we conduct experiments on the SEN1-2 dataset, and the results show that our method performs better compared to the existing methods.

Estimating the ground displacement from non-rigid registration of two optical satellite images, separated from hours to months, is key in the study of natural disasters such as earthquakes. Compared to standard image registration and flow estimation tasks, a key challenge here lies in resolving very small displacements (typically cm- or m-scale) with sub-pixel accuracy and precision using coarser image resolutions (e.g. 15 m for Landsat-8). Traditional block matching/sliding window methods, employing local windowed correlation techniques, are unable to reduce the effects of long-wavelength noise arising from differences in image lightning, vegetation, or acquisition artifacts. By using both local and global scales, fully convolutional deep learning registration models (U-nets) are potentially able to better resolve ground displacements, less affected my multi-scale noise. Yet, no labelled database exists for ground deformation. Here, we develop a new synthetic database of 50,000 realistic satellite image pairs containing simulated earthquake displacements, along with their ground truth displacement maps, which are used to train state-of-the-art fully convolutional deep learning models (U-net). The inference shows good preliminary results, with a fast computation time (less than 1 second for a 256 × 256 image).

This work focuses on an image fusion protocol that combines conventional RGB cameras with multi-aperture devices utilizing Fabry-Perot interferometry, an unconventional way to acquire hyperspectral data with various advantages compared to dispersive spectrometers. The proposed system aims to enhance hyperspectral imaging quality by preserving high-resolution details from conventional cameras while incorporating the specific spectral range captured by the interferometric device, which mitigates the drawbacks associated to the difficulty of manufacturing Fabry-Perot etalons with low thickness. Envisioned for deployment on unmanned aerial vehicles (UAVs) and microsatellites, this compact setup holds promise for applications in environmental monitoring, agriculture, disaster management, and urban planning. The study presents theoretical foundations, addresses challenges, and offers preliminary results, demonstrating the effectiveness of the proposed approach through Bayesian inference.

Over the last few years, several works have proposed deep learning architectures to learn dynamical systems from observation data with no or little knowledge of the underlying physics. A line of work relies on learning representations where the dynamics of the underlying phenomenon can be described by a linear operator, based on the Koopman operator theory. However, despite being able to provide reliable long-term predictions for some dynamical systems in ideal situations, the methods proposed so far have limitations, such as requiring to discretize intrinsically continuous dynamical systems, leading to data loss, especially when handling incomplete or sparsely sampled data. Here, we propose a new deep Koopman framework that represents dynamics in an intrinsically continuous way, leading to better performance on limited training data, as exemplified on several datasets arising from dynamical systems.

Spectral information of the scene can be reconstructed from processing observations acquired by interferometric devices. In the case of devices that have multiple wave interference (e.g., Fabry-Pérot etalons), a simple inversion such as inverse Fourier transform (e.g., for Michelson-like interferometers) of the measured interferograms is not straightforward due to the ill-posedness of the problem. In this paper, we represent the system through an ∞-wave model. The spectral reconstruction is done by a model-based approach as we have a good knowledge of the system. Specifically, we propose to use Loris-Verhoeven algorithm with proximal solvers and induced sparsity on the Fourier domain of the desired spectrum. Our proposal is more robust to noise compared to conventional reconstruction algorithms, as demonstrated by experiments that are carried with interferograms computed from real spectral acquisitions.

Ideally, evaluation of panchromatic and multispectral image fusion requires the use of a reference image, which is only available in a reduced scale protocol. Thus, no reference metrics such as the now standard Quality No Reference (QNR) were introduced. However, the QNR contains implicit implementation parameters which have not been studied yet. Using a statistical analysis based on rank correlation, we show that those parameters have a significant effect on the QNR values. Moreover, we extend previous results indicating that the QNR has low correlation with reference metrics at reduced scale. These results raise questions about the QNR’s relevance. They call for a standardization of implicit parameters so as to compare values across works, and for the QNR to only be seen as a complementary measure to reference metrics but not as a proxy. The developed protocol is also used to find the best set of implicit parameters, but could be generalized for the assessment of other no reference metrics. Finding such well behaved no reference metric is of critical interest for the development of unsupervised machine learning methods of pansharpening.

Detecting and characterizing slow slip events (SSEs) in Global Navigation Satellite System (GNSS) time series is challenging and multi-station deep-learning approaches are still little explored. The main difficulty is the high level of noise in GNSS time series. The noise affecting GNSS measurements is spatially and temporally correlated, which requires setting up multi-station methods to better characterize the spatial extent of slow slip events. Here, we develop and compare different deep-learning approaches to detect and characterize SSEs in GNSS data, showing that methods embedding the spatial information outperform time-series-based approaches, with spatiotemporal models being the most promising and flexible on real GNSS data.

The utilization of hyperspectral imaging in remote sensing has seen an increasing trend, as it enables to capture a greater amount of information. In this context, emerging snapshot sensors based on compressed sensing have been employed for various remote sensing applications. This work presents a prospective study by proposing a method to evaluate the performances we can expect when reconstructing data from a compressed sensing imager, the Double-Disperser Coded Aperture Snapshot Spectral Imager on an embedded system, i.e. on either a Graphics Processing Unit or a Field-Programmable Gate Arrays. This is original in the literature since most compressive sensing works focus on reconstruction quality and overlook the requirements for real-time, namely computational cost and data bandwidth. Moreover, works that use an embedded system are even more scarse. The study introduces methods to enhance these restrictions and assesses the resulting improvements. The study’s findings support the use of Disperser Coded Aperture Snapshot Spectral Imager for remote sensing applications, potentially enabling a smaller sensor size.

Image correlation is a powerful method for remotely constraining ground displacements associated with natural disasters. By employing sub-pixel correlation algorithms, one can obtain a displacement field by correlating satellite images acquired before and after a displacement event. However, this computation may be biased when dealing with sharp discontinuities, typical of earthquake surface ruptures, which are of current interest in the context of quantifying the partitioning of slip between the primary fault core and neighboring damage zone. In this paper, we present an innovative deep learning method to perform sub-pixel correlation of optical satellite images for the retrieval of ground displacement, designed to mitigate bias around fault ruptures. From the generation of a realistic simulated database of images before and after synthetic ground displacement built specifically to deal with fault discontinuities in satellite images (e.g. Landsat-8 in this case), we developed a Convolutional Neural Network (CNN) able to retrieve sub-pixel displacements. Comparison with a state-of-the-art phase correlation method shows our pipeline is able to mitigate the sub-pixel bias in the near-field of earthquake ruptures.

Microsatellites and drones are often equipped with digital cameras whose sensing system is based on color filter arrays (CFAs), which define a pattern of color filter overlaid over the focal plane. Recent commercial cameras have started implementing RGBW patterns, which include some filters with a wideband spectral response together with the more classical RGB ones. This allows for additional light energy to be captured by the relevant pixels and increases the overall SNR of the acquisition. Demosaicking defines reconstructing a multi-spectral image from the raw image and recovering the full color components for all pixels. However, this operation is often tailored for the most widespread patterns, such as the Bayer pattern. Consequently, less common patterns that are still employed in commercial cameras are often neglected. In this work, we present a generalized framework to represent the image formation model of such cameras. This model is then exploited by our proposed demosaicking algorithm to reconstruct the datacube of interest with a Bayesian approach, using a total variation regularizer as prior. Some preliminary experimental results are also presented, which apply to the reconstruction of acquisitions of various RGBW cameras.

This paper proposes a new loss function to train a convolutional neural network (CNN) for multispectral and hyper-spectral (MS-HS) image fusion. The loss function is based on the relative dimensionless global error synthesis (ER-GAS), where we exchange the mean squared error (MSE) for its unbiased estimate using Stein’s risk unbiased estimate (SURE). The loss function has a good balance between the spectral and spatial information implied by the weighted MSE, therefore it does not need a parameter to balance the spectral and spatial terms as in MSE loss function, and it also converges faster than the MSE one. Additionally, the loss function enables unsupervised training and avoids overfit-ting, since it is derived by using SURE. Experimental results show that the proposed method yields good results and outperforms the competitive methods. Codes are available at https://github.com/hvn2/SURE-ERGAS

Multispectral remote sensing images are often have band-dependent image resolution due to cost and technical limitations. To address this, we developed a method that sharpens low-resolution (LR) images using high-resolution (HR) images. In this paper, we propose a novel unsupervised deep learning (DL) approach that involves unrolling an iterative algorithm into a deep neural network and training it using a loss function based on Stein’s risk unbiased estimate (SURE) to sharpen the LR bands (20 and 60 m) of Sentinel-2 (S2) to their highest resolution (10 m). This approach views traditional optimization model-based methods through a DL framework, improving interpretability and clarifying connections between the two approaches. Results from both simulated and real S2 datasets demonstrate that the proposed method outperforms competitive methods and produces high-quality images for the 20 m and 60 m bands. The codes are available at: https://github.com/hvn2/S2-Unrolling.

Target detection based on hyperspectral image (HSI) representations has drawn wide attention given its wide variety of features. The matrix-based approach inevitably loses spatial information and fails to explore the intrinsic multimodal structure of an HSI cube. In this paper, we propose a regularized tensor-based model without altering the data structure. We assume that an observed third-order HSI tensor is decomposed into the sum of a Total Variation regularized Low-rank background tensor and a Sparse (TVLrS) target tensor. The two tensors are represented as the mode-3 product of a third-order tensor, called the Tensor Representation Coefficient (TRC), and a spectra dictionary matrix. Then, the model is coined as TVLrS-TRC. The background TRC has a low-rank property, contributing to the low-rankness characterization in our model. Moreover, as the size of the background TRC term is smaller than the background tensor, characterizing its local smoothness via TV regularization reduces the computational cost compared to that of the background tensor. Extensive experiments on two real hyperspectral datasets demonstrate the advantage of the proposed method compared with the state-of-the-art.

Hyperspectral unmixing allows to represent mixed pixels as a set of pure materials weighted by their abundances. Spectral features alone are often insufficient, so it is common to rely on other features of the scene. Matrix models become insufficient when the hyperspectral image is represented as a high-order tensor with additional features in a multimodal, multi-feature framework. Tensor models such as Canonical polyadic decomposition allow for this kind of unmixing, but lack a general framework and interpretability of the results. In this paper, we propose an interpretable methodological framework for low-rank Multi-feature hyperspectral unmixing based on tensor decomposition (MultiHU-TD) which incorporates the abundance sum-to-one constraint in the Alternating optimization ADMM algorithm, and provide in-depth mathematical, physical and graphical interpretation and connections with the extended linear mixing model. As additional features, we propose to incorporate mathematical morphology and reframe a previous work on neighborhood patches within MultiHU-TD. Experiments on real hyperspectral images showcase the interpretability of the model and the analysis of the results. Python and MATLAB implementations are made available on GitHub.

Monitoring vegetation growth, phenology and health in agriculture requires very high spatial and spectral resolution sensors. Sentinel-2 is among the spaceborne sensors that tried to meet these requirements. However, due to physical constraints, few visible bands are present with relatively large spectral responses which limits the leaf pigment content estimation. In this work, we propose to fuse Sentinel-2 bands with a Sentinel-3 image which has a lower spatial resolution but contains several spectral narrower bands in both the visible and near-infrared domains. The fusion procedure consists in sharpening the Sentinel-3 bands to match the Sentinel-2 spatial resolution leading to a higher spatial resolution Sentinel-3 image. The proposed fusion technique follows a physics-based approach based on the use of a radiative transfer model for establishing a correspondence between Sentinel-2 and Sentinel-3 images. In greater details, the main spectrally pure constituent of each high resolution Sentinel-2 pixel is identified for each pixel and it is then related to the corresponding low resolution Sentinel-3 pixel. Pure materials are the barycenters of clusters obtained by an unsupervised classification of the Sentinel-2 image. Sentinel-3 signatures are obtained using coarse resolution pixel matching with the Sentinel-2 image. The latter result is then corrected using radiative transfer modeling allowing the production of more realistic signatures. Validation was done using real Sentinel-2/Sentinel-3 images taken with a delay of only one day or less and by comparing the sharpened Sentinel-3 bands Oa04, Oa06, Oa08, and Oa17 with the corresponding Sentinel-2 bands B2, B3, B4, and B8A, as they are respectively spectrally close. For all the bands, the RMS is lower than 0.007, and compared to the state-of-the-art techniques it shows competitive results and robustness against scene heterogeneity. Besides, our findings prove that the retrieved Sentinel-3 signatures at full resolution are physically consistent and in good agreement with the Sentinel-2 data.

Hyperspectral anomaly detection (AD) task is a typical binary classification problem, and utilizing background prior knowledge is a key technique to solving such problems. The two most commonly used priors for hyperspectral images are low-rank and local smooth properties. Most traditional matrix-based methods use two regularizations to model these two types of priors and integrate them into one model, which makes these two regularizations unable to maximize their effectiveness. In addition, the matrix method also destroys the structure of the hyperspectral images (HSI). To address these issues, this study identified a unique sparsity property in the gradient tensor of HSI. Specifically, the core tensor resulting from the Tucker decomposition of the gradient tensor was observed to exhibit sparsity. This sparsity property, referred to as GCS (the sparsity on the core tensor of the gradient map), effectively captures the structural information of HSI and improves detection performance. The GCS regularization offers the following advantages: 1) GCS regularization uses one term to simultaneously capture both low-rankness and local smoothness, the size of the core tensor represents the low-rank prior to the background, and the ℓ1norm describes the sparsity of gradient map, i.e., the local smoothness of the original data; 2) GCS is a constrained regularization, allowing for the full utilization of information from different dimensions of the HSI when updating the core tensor, i.e., utilizing the spatial and spectral information carried by three-factor matrices of the Tucker decomposition. Finally, extensive experiments validate the superiority of our proposed methods.

Precise estimation of ground displacement at regional scales from optical satellite imagery is fundamental for the study of natural disasters, such as earthquakes, volcanoes, landslides, etc. Current methods make use of correlation techniques between two acquisitions in order to retrieve a fractional pixel shift. However, differences in local lighting conditions between two acquisitions can lead to differences in image reflectance, which in turn can bias the displacement estimate, especially in the sub-pixel domain. Data-driven methods may provide a way to overcome these errors. From the generation of a realistic simulated database based on Landsat-8 satellite image pairs with added simulated sub-pixel shifts, we developed a Convolutional Neural Network (CNN) able to retrieve sub-pixel displacements.

There is an increasing industrial and scientific demand for large scale monitoring of gas concentration, e.g., to abide with constraints on pollutants for environmental safety. This paper aims to show the potential of the very recent ImSPOC technology for gas monitoring. We present the ImSPOC operating principle, some of the developed prototypes and two illustrative examples in the monitoring of methane and carbon dioxide in controlled environments.

Recent advances in deep learning (DL) reveal that the structure of a convolutional neural network (CNN) is a good image prior (called deep image prior (DIP)), bridging the model-based and DL-based methods in image restoration. However, optimizing a DIP-based CNN is prone to over-fitting leading to a poorly reconstructed image. This paper derives a loss function based on Stein's unbiased risk estimate (SURE) for unsupervised training of a DIP-based CNN applied to the hyperspectral image (HSI) super-resolution. The SURE loss function is an unbiased estimate of the mean-square-error (MSE) between the clean low-resolution image and the low-resolution estimated image, which relies only on the observed low-resolution image. Experimental results on HSI show that the proposed method not only improves the performance, but also avoids overfitting. Codes are available at https://github.com/hvn2/SURE-MS-HS

Image fusion is utilized in remote sensing due to the limitation of the imaging sensor and the high cost of simultaneously acquiring high spatial and spectral resolution images. Optical remote sensing imaging systems usually provide images of high spatial resolution but low spectral resolution and vice versa. Therefore, fusing those images to obtain a fused image having both high spectral and spatial resolution is desirable in many applications. This paper proposes a fusion framework using an unsupervised convolutional neural network (CNN) and Stein’s unbiased risk estimate (SURE). We derive a new loss function for a CNN that incorporates back-projection mean-squared error with SURE to estimate the projected mean-square-error (MSE) between the fused image and the ground truth. The main motivation is that training a CNN with this SURE loss function is unsupervised and avoids overfitting. Experimental results for two fusion examples, multispectral and hyperspectral (MS-HS) image fusion, and multispectral and multispectral (MS-MS) image fusion, show that the proposed method yields high quality fused images and outperforms the competitive methods. Codes are be available at https://github.com/hvn2/Deep-SURE-Fusion.

Light detection and ranging (LiDAR) provides high-resolution geometric information for monitoring forests at individual tree crown (ITC) level. An important task for ITC delineation is segmentation, and previous studies showed that the adaptive 3-D mean shift (AMS3D) algorithm provides effective results. AMS3D for ITC segmentation has three components for the kernel profile: shape, weight, and size. In this letter, we present an AMS3D approach based on the adaptation of the kernel profile size through an ellipsoid crown shape model. The algorithm parameters are estimated based on allometry equations derived from 22 forest plots in two study sites. After computing the mean shift (MS) vector, we initialize the parameters of the ellipsoid crown shape model to derive the kernel profile size, and further tested two crown shape models for adapting the size of the superellipsoid (SE) kernel profile. These schemes are compared with two other MS algorithms with and without kernel profile size adaptation. We select the best algorithm output per plot based on the maximum F1-score. The ellipsoid crown shape model with a SE kernel profile of$n = 1.5$presents the highest recall and the best Jaccard index, especially for conifers.

Although multimodal remote sensing data analysis can strongly improve the characterization of physical phenomena on Earth's surface, nonidealities and estimation imperfections between records and investigation models can limit its actual information extraction ability. In this article, we aim at predicting the maximum information extraction that can be reached when analyzing a given data set. By means of an asymptotic information theory-based approach, we investigate the reliability and accuracy that can be achieved under optimal conditions for multimodal analysis as a function of data statistics and parameters that characterize the multimodal scenario to be addressed. Our approach leads to the definition of two indices that can be easily computed before the actual processing takes place. Moreover, we report in this article how they can be used for operational use in terms of image selection in order to maximize the robustness of the multimodal analysis, as well as to properly design data collection campaigns for understanding and quantifying physical phenomena. Experimental results show the consistency of our approach.

In manufacturing, the monitoring of the fabrication process is crucial in order to be sure that objects are compliant. For nano-objects, most of this monitoring is done manually. In this paper, we propose a method to segment different materials in a manufactured object. The method uses design information which represent the ideal object to manufacture. This representation visually gathers information about materials, shapes and relationships between these shapes. In our segmentation method we choose to encode this information in the tree of shapes to enforce the design characteristics into a real image of the object. To achieve such segmentation, we perform graph cuts on this particular tree structure using additional information such as the position in the design or the order of inclusion of the shapes.

Hyperspectral image unmixing is an inverse problem aiming at recovering the spectral signatures of pure materials of interest (called endmembers) and estimating their proportions (called abundances) in every pixel of the image. However, in spite of a tremendous applicative potential and the avent of new satellite sensors with high temporal resolution, multitemporal hyperspectral unmixing is still a relatively underexplored research avenue in the community, compared to standard image unmixing. In this paper, we propose a new framework for multitemporal unmixing and endmember extraction based on a state-space model, and present a proof of concept on simulated data to show how this representation can be used to inform multitemporal unmixing with external prior knowledge, or on the contrary to learn the dynamics of the quantities involved from data using neural network architectures adapted to the identification of dynamical systems.

This study focuses on a novel snapshot Fourier Transform imaging spectrometer based on an array of Fabry-Perot interferometers. This device fully relies on signal processing in order to provide intelligible outputs and thus requires a precise characterisation. In this paper, we present a strategy for estimating the thickness of the Fabry-Perot cavities, as this information is typically not precise or even available. This is a fundamental step for obtaining calibrated acquisitions with this device and for allowing further data processing and analysis. The proposed technique relies on the device optical model and has proven effective with respect to alternative strategies when applied to real acquisitions.

Sub-pixel mapping (SPM) denotes a category of image processing techniques that further enhance the results provided by spectral unmixing algorithms. While the latter is only capable to determine the fractional abundances of classes with an associated spectral signature within a certain area denoted as mixed pixel, SPM can in addition spatially locate each class separately within the mixed pixel itself, enhancing the spatial resolution of its products. Given the demands by both technological and scientific application, various approaches have been proposed; this work will focus on the ones belonging to the domain of variational frameworks, which allow to solve the intrinsic ill-posedness of inverse problems by imposing a regularization based on gradients of the desired output. As the problem of SPM may be also seen as generating a rule to associate each mixed pixel to a specific patch of mosaicked classes, we propose to create an overcomplete dictionary listing them. We present here some first investigation in this direction, by incorporating such dictionary, generated via K-SVD dictionary learning algorithm, in the formulation of an inverse problem. The algorithm is matched with a Isotropic Total Variation (ITV) regularization to provide joint spatial and spectral consistency to the results. Our tests prove that our proposed method provides better SPM product quality compared to its use in insulation and to other state of the art variational framework-based algorithms.

Leaf angle distribution is an important phenotype parameter that is related to photosynthesis. Thanks to the recent advent of drones and high-resolution imaging devices, leaf-scale aerial images with high spectral and spatial resolution are available. This work is the first attempt to utilize a single leaf-scale image to differentiate plants with different leaf angle distribution. First, assuming that a rice leaf surface resembles a section of a hemiellipsoid surface, a collection of rice leaf surfaces is approximated by a hemiellipsoid surface. Time-series of shading distributions on the hemiellipsoids with different structural parameters under different direct sunlight directions are generated. By investigating the statistical properties, i.e., skewness, kurtosis and the most probable intensity, of the frequencies of the simulated shading intensity that well-differentiate hemiellipsoids with different structural parameters, we identified an appropriate time slot, i.e., 11: 00-12:30, for image acquisitions. Then, time-series leaf-scale images and depth maps of rice plants with/without silicate fertilizer under sunlight were collected. Based on the depth maps, it was confirmed that silicate fertilizer dosed leaves are more upright than leaves from non treated plants. It was demonstrated that 89% and 100% of kurtosis and the most probable intensity of the leaf-scale images during the appropriate time slot showed consistent relations with the simulations, which indicates that the proposed method is useful to distinguish different leaf angle distributions based on the frequency of shading intensity of rice leaf images.

Image classification has been at the core of remote sensing applications. Optical remote sensing imaging systems naturally acquire images with spectral features corresponding to pixels. Spectral classification ignores the spatial distribution of the data which is becoming more relevant with the development of spatial resolution sensors, and many works aim to incorporate spatial features based on neighborhood through for example, Mathematical Morphology (MM). Additionally, one could stack multiple morphological transformations of the image resulting in a highly complex block of data. Since classification is a tool that requires a matrix of samples and features, and simply stacking the different sets of features can lead to the problem of high dimensionality, we propose a way to create a matrix of low dimensional feature space by modeling the data as tensors and thanks to Canonical Polyadic (CP) decomposition. Experiments on real image show the effectiveness of the proposed method.

Hyperspectral imaging is an important source of land cover information by virtue of its spectral richness. However, this type of imagery is typically known by its coarse spatial resolution, that is a limiting factor for end-users. Although spectral unmixing techniques can provide subpixellic information by means of abundance fractions for each class in mixed pixels, the spatial distribution of these classes within each pixel is still unknown. Sub-pixel mapping techniques address the above mentioned problem. Nevertheless, the traditional sub-pixel mapping algorithms based on spatial dependence assumptions cannot solve these problems efficiently. Spatial regularization methods have recently been proposed in a way that they can treat each abundances map separately and do not consider spatial correlation between classes. In order to improve sub-pixel mapping accuracy and, consequently, enhance hyperspectral image classification, we propose a sub-pixel mapping method based on isotropic total variation minimization within and between pixels for different classes simultaneously. Experimental results with synthetic data sets show the attributes of using total variation as a prior model, which leads to improve sub-pixel mapping of different classes together.

Among direct and diffuse components that compose photosynthetically active radiation (PAR), diffuse component of sunlight is important to evaluate fraction absorbed PAR (FAPAR) of plants because diffuse flux penetrates more deeply than direct flux and the peak of photosynthetic photon flux density (PPFD) use efficiency occurs at low to medium PPFD. Shading distribution in leaf-scale aerial images of plants by low-altitude measurement via UAVs contains sunlight information as well as plant structure and leaf pigments. In this work, we investigate the relationship between statistical properties of leaf-scale images, share of diffuse flux (SDF) in global radiation and solar zenith angles (SZA). Higher-order statistics (HOSs) were calculated from ground-based close-range images of wheat leaves under various sunlight conditions. SDF were measured based on two field spectrometers. We confirmed that (1) images under clear sky can be distinguished from those under cloudy sky based on HOSs, and (2) it is possible to estimate SZA based on HOSs of leaf-scale images under clear sky.

A method based on spectral unmixing (SU) for snow cover estimation from a time series of optical images is proposed in this letter. Specifically, we have developed an endmember estimation procedure that exploits the temporal continuity of a scene. Consecutive dates are jointly processed for a more precise description of background materials improving the estimation of a fractional snow cover map. The proposed workflow relies on up to three consecutive acquisitions over the same area to extract an appropriate set of background spectra to be considered as endmembers. In greater details, multiple sets of endmembers are extracted from different images in the time series by a geometrical automated endmember extraction algorithm and the most relevant one is selected in terms of reconstruction error. Snow cover maps are then estimated by SU considering as endmembers the snow spectra coming from a spectral library and those associated with the background materials as estimated by the proposed procedure. The proposed technique is quantitatively validated considering Moderate-Resolution Imaging Spectroradiometer Terra data over the French Alps and Moroccan High Atlas image time series and comparing the estimated snow cover maps with high-resolution reference data. The experiment clearly demonstrates the effectiveness of the generated set of endmembers using three different approaches to abundance estimation.

Semantic segmentation requires methods capable of learning high-level features while dealing with large volume of data. Toward such goal, convolutional networks can learn specific and adaptable features based on the data. However, these networks are not capable of processing a whole remote sensing image, given its huge size. To overcome such limitation, the image is processed using fixed size patches. The definition of the input patch size is usually performed empirically (evaluating several sizes) or imposed (by network constraint). Both strategies suffer from drawbacks and could not lead to the best patch size. To alleviate this problem, several works exploited multicontext information by combining networks or layers. This process increases the number of parameters, resulting in a more difficult model to train. In this paper, we propose a novel technique to perform semantic segmentation of remote sensing images that exploits a multicontext paradigm without increasing the number of parameters while defining, in training time, the best patch size. The main idea is to train a dilated network with distinct patch sizes, allowing it to capture multicontext characteristics from heterogeneous contexts. While processing these varying patches, the network provides a score for each patch size, helping in the definition of the best size for the current scenario. A systematic evaluation of the proposed algorithm is conducted using four high-resolution remote sensing data sets with very distinct properties. Our results show that the proposed algorithm provides improvements in pixelwise classification accuracy when compared to the state-of-the-art methods.

Recently, the Sentinel-2 (S2) satellite constellation was deployed for mapping and monitoring the Earth environment. Images acquired by the sensors mounted on the S2 platforms have three levels of spatial resolution: 10, 20, and 60 m. In many remote sensing applications, the availability of images at the highest spatial resolution (i.e., 10 m for S2) is often desirable. This can be achieved by generating a synthetic high-resolution image through data fusion. To this end, researchers have proposed techniques exploiting the spectral/spatial correlation inherent in multispectral data to sharpen the lower resolution S2 bands to 10 m. In this paper, we propose a novel method that formulates the sharpening process as a solution to an inverse problem. We develop a cyclic descent algorithm called S2Sharp and an associated tuning parameter selection algorithm based on generalized cross validation and Bayesian optimization. The tuning parameter selection method is evaluated on a simulated data set. The effectiveness of S2Sharp is assessed experimentally by comparisons to state-of-the-art methods using both simulated and real data sets.

The combination of a multispectral (MS) image and a panchromatic (PAN) image, the so-called pansharpening, allows to produce very appealing images that are useful both for visual interpretation and for feature extraction. The state-of-the-art multiresolution analysis pansharpening algorithms are based on the extraction of spatial details from the PAN image through image filters matched with the MS sensors' modulation transfer function. However, this knowledge is often poor due to measurement inaccuracies and/or its aging. Thus, deconvolution algorithms have been proposed to overcome this limitation. In this paper, we propose a multiband filter estimation (FE) approach to improve the solutions in the literature. The main idea in this paper is to exploit a preliminary pansharpened image to estimate the spatial filter used for detail extraction associated with each spectral band. We demonstrate that the proposed method outperforms the state-of-the-art FE approaches by employing data sets acquired by the IKONOS, the Quickbird, and the WorldView-3 sensors.

In the context of data fusion, pansharpening refers to the combination of a panchromatic (PAN) and a multispectral (MS) image, aimed at generating an image that features both the high spatial resolution of the former and high spectral diversity of the latter. In this work we present a model to jointly solve the problem of data fusion and reconstruction of a compressed image; the latter is envisioned to be generated solely with optical on-board instruments, and stored in place of the original sources. The burden of data downlink is hence significantly reduced at the expense of a more laborious analysis done at the ground segment to estimate the missing information. The reconstruction algorithm estimates the target sharpened image directly instead of decompressing the original sources beforehand; a viable and practical novel solution is also introduced to show the effectiveness of the approach.

Multiresolution optical remote sensing systems often have a spatial resolution that varies between bands. An example is the Sentinel-2 (S2) constellation which has three levels of spatial resolution 10m, 20m, and 60m. Recently, researchers have exploited the spectral/spatial correlation inherent in multispectral data to sharpen the lower resolution S2 bands. In this paper, we propose a low-rank method that formulates the sharpening process as a solution to a cost function. We develop an iterative algorithm based on cyclic descent and call it S2Sharp-CD. We evaluate the method on a simulated dataset and compare it to a state-of-the-art approach.

There is an inevitable trade-off between spatial and spectral resolutions in optical remote sensing images. A number of data fusion techniques of multimodal images with different spatial and spectral characteristics have been developed to generate optical images with both spatial and spectral high resolution. Although some of the techniques take the spectral and spatial blurring process into account, there is no method that attempts to retrieve an optical image with both spatial and spectral high resolution, a spectral blurring filter and a spectral response simultaneously. In this paper, we propose a new framework of spatial resolution enhancement by a fusion of multiple optical images with different characteristics based on tensor decomposition. An optical image with both spatial and spectral high resolution, together with a spatial blurring filter and a spectral response, is generated via canonical polyadic (CP) decomposition of a set of tensors. Experimental results featured that relatively reasonable results were obtained by regularization based on nonnegativity and coupling.

Hyperspectral video sequences (HVSs) are well suited for gas plume detection (GPD). The high spectral resolution allows the detection of chemical clouds even when they are optically thin. Processing this new type of video sequences is challenging and requires advanced image and video analysis algorithms. In this paper, we propose a novel method for GPD recorded in HVSs. Based on the assumption that the background is stationary and the gas plume is moving, the proposed method separates the background from the gas plume via a low-rank and sparse decomposition. Furthermore, taking into consideration that the gas plume is continuous in both spatial and temporal dimensions, we include total variation regularization in the constrained minimization problem, which we solve using the augmented Lagrangian multiplier method. After applying the above process to each extracted feature, a novel fusion strategy is proposed to combine the information into a final detection result. Experimental results using real data sets indicate that the proposed method achieves very promising GPD performance.

Pansharpening is an important application in remote sensing image processing. It can increase the spatial-resolution of a multispectral image by fusing it with a high spatial-resolution panchromatic image in the same scene, which brings great favor for subsequent processing such as recognition, detection, etc. In this paper, we propose a continuous modeling and sparse optimization based method for the fusion of a panchromatic image and a multispectral image. The proposed model is mainly based on reproducing kernel Hilbert space (RKHS) and approximated Heaviside function (AHF). In addition, we also propose a Toeplitz sparse term for representing the correlation of adjacent bands. The model is convex and solved by the alternating direction method of multipliers which guarantees the convergence of the proposed method. Extensive experiments on many real datasets collected by different sensors demonstrate the effectiveness of the proposed technique as compared with several state-of-the-art pansharpening approaches.

Most of the tests proposed in the literature to verify if a given random multivariate dataset fits a spherical or elliptical distribution are designed for real valued data and rely on the estimation of high order moment matrices. Recently, a test that considers complex random vectors, derived based on the Schott spherical symmetry test was proposed aiming in a more proper analysis of PolSAR data. Results showed its effectiveness in discriminating data that fits or not the complex spherically invariant random vector model (product model), inherent to high resolution heterogeneous PolSAR systems. Within this context, this paper further extends the assessment of the referred test efficiency, verifying its performance under different stochastic model assumptions and comparing the results with the ones achieved when the Schott test derived for real random vectors is employed.

Inpainting in hyperspectral imagery is a challenging research area and several methods have been recently developed to deal with this kind of data. In this paper we address missing data restoration via a convex optimization technique with regularization term based on Collaborative Total Variation (CTV). In particular we evaluate the effectiveness of several instances of CTV in conjunction with different dimensionality reduction algorithms.

In this paper, we propose a continuous modeling and sparse optimization based method for the fusion of a panchromatic (PAN) image and a multispectral (MS) image. The proposed model is mainly based on reproducing kernel Hilbert space (RKHS) and approximated Heaviside function (AHF). In addition, we also design an iterative strategy to recover more image details. The final model is a convex one and solved by the designed alternating direction method of multipliers (ADMM) which guarantees the convergence of the proposed method. Experimental results on two real datasets corresponding to different sensors and different resolutions demonstrate the effectiveness of the proposed approach as compared with several state-of-the-art pansharpening approaches.

Variational methods are widely used in image processing for problems ranging from denoising to data fusion. In this paper we focus on a recent regularization method, called Collaborative Total Variation, applied to the hyperspectral pansharpening, which deals with the fusion of low resolution hyperspectral and high resolution panchromatic images. The effectiveness of this novel approach is evaluated for different Collaborative Norms and the assessment is performed on the Pavia University dataset.

Self-Dual Attribute Profiles (SDAPs) have proven to be an effective method for extracting spatial features able to improve scene classification of remote sensing images with very high spatial resolution. An SDAP is a multilevel decomposition of an image obtained with a sequence of transformations performed by attribute filters over the Tree of Shapes (ToS). One of the main issues with this technique is the identification of the filter thresholds generating a SDAP composed of features that should be relevant for the classification problem. This paper proposes a tree-based supervised feature extraction strategy, which is based on Fisher's linear discriminant analysis relying on the available class information. The exploitation of the ToS structure in the threshold selection procedure allows one to avoid any prior full image filtering, as in other related techniques. Furthermore, the ToS automates and optimizes the whole process by decreasing the computational time and overcoming the conventional selection procedure based on trial and error attempts. The proposed automatic spatial feature extraction technique has been tested in the classification of a very high resolution image proving its effectiveness with respect to a conventional selection strategy.

We propose to use the temporal coherence of a time series to extract using Vertex Component Analysis (VCA) the suitable set of endmembers for each scene. The reconstruction error computed on the two previous scenes for each date is used to constrain the selection of the set of endmembers produced by VCA. Snow cover estimation is considered as application. We tested different approaches for abundance estimation (FCLSU, SUnSAL, ELMM) over the French Alps from Moderate Resolution Imaging Spectroradiometer (MODIS) images. Results shows a decrease of the false positive rate with the proposed approach.

Cloude and Pottier H/α feature space [1] is one of the most employed methods for unsupervised PolSAR data classification based on Incoherent Target Decomposition. The association of the coherence matrix eigenvectors to the most dominant scatters in the analysed pixel introduces unfeasible regions in the H/α plane. The Independent Component Analysis provides promising new information to better interpret non-Gaussian heterogeneous clutter in the frame of polarimetric incoherent target decompositions. Not constrained to any orthogonality between the estimated scattering mechanisms that compose the clutter under analysis, ICA does not introduce any unfeasible region in the H/α plane, increasing the range of possible natural phenomenons depicted in the aforementioned feature space.

Classical pansharpening algorithms constitute a class of image fusion methods that have been widely investigated in the literature. They have been developed for combining a single- and a multichannel image (panchromatic (PAN) and multispectral (MS), respectively), but can be adapted to the sharpening of hyperspectral (HS) data, both through companion PAN and MS images. We focus in this letter on the HS/MS fusion, showing that the assignation of the MS channel to each HS band is a key step, and investigate several alternatives to make this choice. The assignment algorithms are tested in conjunction with both component substitution and multiresolution analysis pansharpening methods and assessed on images acquired by the Hyperion and ALI sensors. The numerical evaluation shows that the best results can be obtained by optimizing the spectral angle mapper metric confirming that classical methods represent a reliable basis for the development of novel sharpening algorithms.

With the development of hyperspectral sensors, nowadays, we can easily acquire large amount of hyperspectral images (HSIs) with very high spatial resolution, which has led to a better identification of relatively small structures. Owing to the high spatial resolution, there are much less mixed pixels in the HSIs, and the boundaries between these categories are much clearer. However, the high spatial resolution also leads to complex and fine geometrical structures and high inner-class variability, which make the classification results very “noisy.” In this paper, we propose a multimorphological superpixel (MMSP) method to extract the spectral and spatial features and address the aforementioned problems. To reduce the difference within the same class and obtain multilevel spatial information, morphological features (multistructuring element extended morphological profile or multiattribute filter extended multi-attribute profiles) are first obtained from the original HSI. After that, simple linear iterative clustering segmentation method is performed on each morphological feature to acquire the MMSPs. Then, uniformity constraint is used to merge the MMSPs belonging to the same class which can avoid introducing the information from different classes and acquire spatial structures at object level. Subsequently, mean filtering is utilized to extract the spatial features within and among MMSPs. At last, base kernels are obtained from the spatial features and original HSI, and several multiple kernel learning methods are used to obtain the optimal kernel to incorporate into the support vector machine. Experiments conducted on three widely used real HSIs and compared with several well-known methods demonstrate the effectiveness of the proposed model.

The Cloude and Pottier H/α feature space is one of the most employed methods for unsupervised polarimetric synthetic aperture radar (PolSAR) data classification based on incoherent target decomposition (ICTD). The method can be split in two stages: the retrieval of the canonical scattering mechanisms present in an image cell and their parameterization. The association of the coherence matrix eigenvectors to the most dominant scattering mechanisms in the analyzed pixel introduces unfeasible regions in the H/α plane. This constraint can compromise the performance of detection, classification, and geophysical parameter inversion algorithms that are based on the investigation of this feature space. The independent component analysis (ICA), recently proposed as an alternative to eigenvector decomposition, provides promising new information to better interpret non-Gaussian heterogeneous clutter (inherent to highresolution SAR systems) in the frame of polarimetric ICTDs. Not constrained to any orthogonality between the estimated scattering mechanisms that compose the clutter under analysis, ICA does not introduce any unfeasible region in the H/α plane, increasing the range of possible natural phenomena depicted in the aforementioned feature space. This paper addresses the potential of the new information provided by the ICA as an ICTD method with respect to Cloude and Pottier H/α feature space. A PolSAR data set acquired in October 2006 by the E-SAR system over the upper part of the Tacul glacier from the Chamonix Mont Blanc test site, France, and a RAMSES X-band image acquired over Brétigny, France, are taken into consideration to investigate the characteristics of pixels that may fall outside the feasible regions in the H/α plane that arise when the eigenvector approach is employed.

Pansharpened images are widely used synthetic representations of the Earth surface characterized by both a high spatial resolution and a high spectral diversity. They are usually generated by extracting spatial details from a high-resolution PANchromatic image and by injecting them into a low spatial resolution multispectral image. The details injection is performed through injection coefficients, whose values can be either uniform for the whole image (global methods) or spatially variant (context-adaptive (CA) approaches). In this paper, we propose a CA approach in which the injection coefficients are estimated over image segments achieved through a binary partition tree segmentation algorithm. The approach is applied to two credited pansharpening algorithms based on the Gram-Schmidt orthogonalization procedure and the generalized Laplacian pyramid technique. The performance assessment is performed using two different data sets acquired by the QuickBird and the WorldView-3 satellites. The validation procedure, both at full and at reduced resolution, shows the suitability of the proposed approach, which reaches a good tradeoff between accuracy and computational burden.

It is now possible to collect hyperspectral video sequences at a near real-time frame rate. The wealth of spectral, spatial, and temporal information of those sequences is appealing for various applications, but classical video processing techniques must be adapted to handle the high dimensionality and huge size of the data to process. In this paper, we introduce a novel method based on the hierarchical analysis of hyperspectral video sequences to perform object tracking. This latter operation is tackled as a sequential object detection process, conducted on the hierarchical representation of the hyperspectral video frames. We apply the proposed methodology to the chemical gas plume tracking scenario and compare its performances with state-of-the-art methods, for two real hyperspectral video sequences, and show that the proposed approach performs at least equally well.

Morphological attribute profiles are multilevel decompositions of images obtained with a sequence of transformations performed by connected operators. They have been extensively employed in performing multi-scale and region-based analysis in a large number of applications. One main, still unresolved, issue is the selection of filter parameters able to provide representative and non-redundant threshold decomposition of the image. This paper presents a framework for the automatic selection of filter thresholds based on Granulometric Characteristic Functions (GCFs). GCFs describe the way that non-linear morphological filters simplify a scene according to a given measure. Since attribute filters rely on a hierarchical representation of an image (e.g., the Tree of Shapes) for their implementation, GCFs can be efficiently computed by taking advantage of the tree representation. Eventually, the study of the GCFs allows the identification of a meaningful set of thresholds. Therefore, a trial and error approach is not necessary for the threshold selection, automating the process and in turn decreasing the computational time. It is shown that the redundant information is reduced within the resulting profiles (a problem of high occurrence, as regards manual selection). The proposed approach is tested on two real remote sensing data sets, and the classification results are compared with strategies present in the literature.

Thanks to the fast development of sensors, it is now possible to acquire sequences of hyperspectral images. Those hyperspectral video sequences (HVS) are particularly suited for the detection and tracking of chemical gas plumes. In this paper, we present a novel gas plume detection method. It is based on the decomposition of the sequence into a low-rank and a sparse term, corresponding to the background and the plume, respectively, and incorporating temporal consistency. To introduce spatial continuity, a post processing is added using the Total Variation (TV) regularized model. Experimental results on real hyperspectral video sequences validate the effectiveness of the proposed method.

Land cover classification is a task that requires methods capable of learning high-level features while dealing with high volume of data. Overcoming these challenges, Convolutional Networks (ConvNets) can learn specific and adaptable features depending on the data while, at the same time, learn classifiers. In this work, we propose a novel technique to automatically perform pixel-wise land cover classification. To the best of our knowledge, there is no other work in the literature that perform pixel-wise semantic segmentation based on data-driven feature descriptors for high-resolution remote sensing images. The main idea is to exploit the power of ConvNet feature representations to learn how to semantically segment remote sensing images. First, our method learns each label in a pixel-wise manner by taking into account the spatial context of each pixel. In a predicting phase, the probability of a pixel belonging to a class is also estimated according to its spatial context and the learned patterns. We conducted a systematic evaluation of the proposed algorithm using two remote sensing datasets with very distinct properties. Our results show that the proposed algorithm provides improvements when compared to traditional and state-of-the-art methods that ranges from 5 to 15% in terms of accuracy.

Satellite image classification is a key task used in remote sensing for the automatic interpretation of a large amount of information. Today there exist many types of classification algorithms using advanced image processing methods enhancing the classification accuracy rate. One of the best state-of-the-art methods which improves significantly the classification of complex scenes relies on Self-Dual Attribute Profiles (SDAPs). In this approach, the underlying representation of an image is the Tree of Shapes, which encodes the inclusion of connected components of the image. The SDAP computes for each pixel a vector of attributes providing a local multiscale representation of the information and hence leading to a fine description of the local structures of the image. Instead of performing a pixel-wise classification on features extracted from the Tree of Shapes, it is proposed to directly classify its nodes. Extending a specific interactive segmentation algorithm enables it to deal with the multi-class classification problem. The method does not involve any statistical learning and it is based entirely on morphological information related to the tree. Consequently, a very simple and effective region-based classifier relying on basic attributes is presented.

Recent advances in airborne light detection and ranging (LiDAR) technology allow us to rapid measure the topographical information over large areas. LiDAR remote sensed data has been widely used in many applications, e.g. forest management, urban planning, disaster predictions, etc. However, extracting useful information from LiDAR data remains challenging, especially in the urban remote sensing, where many objects have the same elevation and are connected, such as road and parking lots, trees and buildings. In this work, we present a new method to extract geometric and textural information from LiDAR data by using attribute filters with partial reconstruction. The proposed method can separate the connected objects and better model the geometric and textural information than traditional connected filters (e.g. attribute filters). Experimental results on LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using original LiDAR data or attribute profiles computed by traditional attribute filters, with the proposed method, overall classification accuracies were improved by 35% and 12%, respectively.

Thermal Sharpening (TS) is usually referred to techniques widely used in several Earth Observation applications in order to increase the spatial resolution of thermal images. Profiting from the particular design of the Visible Infrared Imaging Radiometer Suite (VIIRS) sensor mounted on board of the Suomi National Polar-orbiting Partnership (NPP) satellite, we propose here a new approach for obtaining synthetic thermal data with increased spatial resolution and spectral diversity. The method exploits classical Pansharpening algorithms, which are very popular in the field of Visible and Near-InfraRed (VNIR) image fusion, for combining the VIIRS thermal bands with partially overlapping spectral responses. We evaluate the effectiveness of several algorithms by performing a Reduced Resolution (RR) assessment on VIIRS real data, showing the importance of an adequate knowledge of the sensor characteristics.

Morphological attribute profiles (APs) are discriminant features in the spectral–spatial classification of hyperspectral data. However, the optimal range of parameters in each filter is always a challenging yet important task, since an unsuitable range of parameters likely leads to inferior results. In order to alleviate this problem, we propose an ensemble method, which integrates multiple classification results based on a series of APs. The APs are obtained by using different filters with thresholds that are randomly selected from an arbitrarily defined range of parameters. Experimental results conducted on two hyperspectral images demonstrate the robustness and effectiveness of the proposed method.

Morphological attribute profiles are among the most prominent spectral-spatial pixel description methods. They are efficient, effective, and highly customizable multiscale tools based on hierarchical representations of a scalar input image. Their application to multivariate images in general and hyperspectral images in particular has been so far conducted using the marginal strategy, i.e., by processing each image band (eventually obtained through a dimension reduction technique) independently. In this paper, we investigate the alternative vector strategy, which consists in processing the available image bands simultaneously. The vector strategy is based on a vector-ordering relation that leads to the computation of a single max and min tree per hyperspectral data set, from which attribute profiles can then be computed as usual. We explore known vector-ordering relations for constructing such max trees and, subsequently, vector attribute profiles and introduce a combination of marginal and vector strategies. We provide an experimental comparison of these approaches in the context of hyperspectral classification with common data sets, where the proposed approach outperforms the widely used marginal strategy.

Remotely sensed images with very high spatial resolution provide a detailed representation of the surveyed scene with a geometrical resolution that, at the present, can be up to 30 cm (WorldView-3). A set of powerful image processing operators have been defined in the mathematical morphology framework. Among those, connected operators [e.g., attribute filters (AFs)] have proven their effectiveness in processing very high resolution images. AFs are based on attributes which can be efficiently implemented on tree-based image representations. In this paper, we considered the definition of min, max, direct, and subtractive filter rules for the computation of AFs over the tree-of-shapes representation. We study their performance on the classification of remotely sensed images. We compare the classification results over the tree of shapes with the results obtained when the same rules are applied on the component trees. The random forest is used as a baseline classifier, and the experiments are conducted using multispectral data sets acquired by QuickBird and IKONOS sensors over urban areas.

The intrinsic dimensionality (ID) of multivariate data is a very important concept in spectral unmixing of hyperspectral images. A good estimation of the ID is crucial for a correct retrieval of the number of endmembers (the spectral signatures of macroscopic materials) in the image, for dimensionality reduction or for subspace learning, among others. Recently, some approaches to perform spectral unmixing and superresolution locally have been proposed, which require a local estimation of the number of endmembers to use. However, the role of ID in local regions of hyperspectral images has not been properly addressed. Some important issues when dealing with small regions of hyperspectral data can seriously affect the performance of conventional hyperspectral ID estimators. We show that three factors mainly affect local ID estimation: the number of pixels in the local regions, which has to be high enough for the estimations to be relevant, the number of hyperspectral bands which complicates the estimations if the ambient space has a high dimensionality, and the noise, which can be misinterpreted as a signal when its power is important. Here, we review the hyperspectral ID estimators on the literature for local ID estimation, we show how they behave in a local setting on synthetic and real data sets, and we provide some guidelines to make proper use of these estimators in local approaches.

Extended attribute profiles (EAPs) have been widely used for the classification of high-resolution hyperspectral images. EAPs are obtained by computing a sequence of attribute operators. Attribute filters (AFs) are connected operators, so they can modify an image by only merging its flat zones. These filters are effective when dealing with very high resolution images since they preserve the geometrical characteristics of the regions that are not removed from the image. However, AFs, being connected filters, suffer the problem of “leakage” (i.e., regions related to different structures in the image that happen to be connected by spurious links will be considered as a single object). Objects expected to disappear at a certain threshold remain present when they are connected with other objects in the image. The attributes of small objects will be mixed with their larger connected objects. In this paper, we propose a novel framework for morphological AFs with partial reconstruction and extend it to the classification of high-resolution hyperspectral images. The ultimate goal of the proposed framework is to be able to extract spatial features which better model the attributes of different objects in the remote sensed imagery, which enables better performances on classification. An important characteristic of the presented approach is that it is very robust to the ranges of rescaled principal components, as well as the selection of attribute values. Our experimental results, conducted using a variety of hyperspectral images, indicate that the proposed framework for AFs with partial reconstruction provides state-of-the-art classification results. Compared to the methods using only single EAP and stacking all EAPs computed by existing attribute opening and closing together, the proposed framework benefits significant improvements in overall classification accuracy.

The multiplicative model, expressed as a product between the square root of a scalar positive quantity (texture) and the description of an equivalent homogeneous surface (speckle), is one of the most appropriate and disseminated models used to describe high-resolution polarimetric synthetic aperture radar (PolSAR) clutter. Generally, the texture is assumed polarization independent, which causes PolSAR data to present a spherical symmetry property, allowing for the usage of most of the algorithms present in the literature. Nevertheless, the existence of polarization-dependent clutter has also been reported, for which specific algorithms need to be derived. Therefore, it becomes clear that the first step in SAR data analysis should be the validation of the model employed. Within this context, this paper presents a new methodological framework to assess the conformity of multivariate high-resolution SAR data with respect to the product model in terms of asymptotic statistics. More precisely, spherical symmetry is investigated by applying statistical hypothesis testing on the structure of the quadricovariance matrix. Simulated data, data from the P-band airborne data set acquired by the Office National d'Études et de Recherches Aérospatiales (ONERA) over the French Guiana in 2009 in the frame of the European Space Agency campaign TropiSAR and a RAMSES X-band image acquired over Brétigny, France, are taken into consideration to investigate the performance of the derived test. The detection results are qualitatively and quantitatively analyzed, and some important conclusions are drawn regarding the methodology employed in analyzing SAR data.

Polarimetric incoherent target decomposition aims at accessing physical parameters of illuminated scatters through the analysis of the target coherence or covariance matrix. In this framework, independent component analysis (ICA) was recently proposed as an alternative method to eigenvector decomposition to better interpret non-Gaussian heterogeneous clutter (inherent to high-resolution synthetic aperture radar systems). Until now, the two main drawbacks reported of the aforementioned method are the greater number of samples required for an unbiased estimation, when compared to the classical eigenvector decomposition, and the inability to be employed in scenarios under the Gaussian clutter assumption. In this paper, both drawbacks are analyzed. First, a Monte Carlo approach is performed in order to investigate the bias in estimating Touzi's target-scattering-vector-model parameters when ICA is employed. Simulated data and a RAMSES X-band image acquired over Brétigny, France, are taken into consideration to investigate the bias estimation under different scenarios. Finally, the performance of the algorithm is also evaluated under the Gaussian clutter assumption and when spatial correlation is introduced in the model.

Remote sensing images exhibit significant contrast and intensity regions and edges, which makes them highly suitable for using different texture features to properly represent and classify the objects that they contain. In this paper, we present a new technique based on multiple morphological component analysis (MMCA) that exploits multiple textural features for decomposition of remote sensing images. The proposed MMCA framework separates a given image into multiple pairs of morphological components (MCs) based on different textural features, with the ultimate goal of improving the signal-to-noise level and the data separability. A distinguishing feature of our proposed approach is the possibility to retrieve detailed image texture information, rather than using a single spatial characteristic of the texture. In this paper, four textural features: content, coarseness, contrast, and directionality (including horizontal and vertical), are considered for generating the MCs. In order to evaluate the obtained MCs, we conduct classification by using both remotely sensed hyperspectral and polarimetric synthetic aperture radar (SAR) scenes, showing the capacity of the proposed method to deal with different kinds of remotely sensed images. The obtained results indicate that the proposed MMCA framework can lead to very good classification performances in different analysis scenarios with limited training samples.

Nonlinear decomposition schemes constitute an alternative to classical approaches for facing the problem of data fusion. In this paper, we discuss the application of this methodology to a popular remote sensing application called pansharpening, which consists in the fusion of a low resolution multispectral image and a high-resolution panchromatic image. We design a complete pansharpening scheme based on the use of morphological half gradient operators and demonstrate the suitability of this algorithm through the comparison with the state-of-the-art approaches. Four data sets acquired by the Pleiades, Worldview-2, Ikonos, and Geoeye-1 satellites are employed for the performance assessment, testifying the effectiveness of the proposed approach in producing top-class images with a setting independent of the specific sensor.

Attribute profiles (APs) have increasingly been receiving more attention over the last years, as they are able to extract and model spatial information that is useful for the analysis of remote sensing images of very high spatial resolution (VHR). However, one of the major issues in employing APs is the choice of a proper range of thresholds, able to provide a representative and non-redundant multi-level image decomposition. This paper presents a novel method for the automatic selection of adequate thresholds to compute the AP. A new concept of cumulative function, which can be seen as an extension of the basic notion of granulometry, is introduced. In particular, different information on the spatial context is achieved according to the measure used for computing the cumulative function, which is computed on the AP composed by considering all possible values of the attribute. The proposed approach aims at selecting the set of thresholds that provides the best approximation of the resulting cumulative function based on the chosen measure. Experimental analysis carried out on a very high resolution image shows the effectiveness of the presented strategy in providing a set of thresholds able to retain the salient spatial structures in the scene.

Pansharpening algorithms enable to produce synthetic data with high spatial details and spectral diversity by combining a panchromatic image with multispectral or hyperspectral data. In classical approaches the details extracted from the panchromatic image are introduced into the original multichannel image through injection gains, which can be spatially variant on the image. In this paper we analyze several methods for partitioning an image into regions in which the pixels will share the same injection coefficients. Gram-Schmidt pansharpening methods are used as paradigmatic examples for assessing the performance of global and local gain estimation strategies, using hyperspectral data acquired by sensors mounted on one (Earth Observing-1) or multiple (PROBA and Quick-bird) satellite platforms.

The Independent Component Analysis (ICA) aims, based on higher order statistical moments, in recovering statistical independent sources and the mixing mechanism, without having any physical background of the latter. Recently proposed as an alternative to Eigenvector decomposition in the analysis of Polarimetric SAR (PolSAR) data, it proved itself to be a very promising tool to better interpret non-Gaussian heterogeneous clutter, being employed in both urban area analysis as well as in snow monitoring applications. In this paper we intend to extend the range of applications of ICA based ICTD by investigating the results and the algorithm performance under tropical forest scenarios. Data from the P-band airborne dataset acquired by the Office National d'Études et de Recherches Aérospatiales (ONERA) over the French Guiana in 2009 in the frame of the European Space Agency campaign TropiSAR is taken into consideration to analyse the potential of supplementary information introduced by the ICA approach.

Pansharpening consists of fusing a multispectral (MS) image together with a panchromatic (PAN) image with the aim of jointly preserving the spectral diversity of the former and the geometric richness of the latter. A crucial step in pansharpening algorithms is the detail extraction. This problem is usually addressed by the means of 2D Gaussian filters matched with the MS sensor's modulation transfer function (MTF). Nevertheless, several issues can affect this characterization (e.g. the MTF's gains at the Nyquist frequency could be not available or unreliable). Thus, in this paper we propose a technique based on blind image deblurring in order to estimate band-dependent spatial detail extraction filters by taking into consideration the possible variability of the MS spatial features along bands. The validation is carried out exploiting two real datasets acquired by the IKONOS and the QuickBird sensors.

In this work, we propose a new multiple morphological component analysis (MMCA) based decomposition framework for remote sensing image classification. The proposed MMCA framework aims at exploiting relevant textural characteristics present in a scene such as content, coarseness, contrast or directionality. Specifically, MMCA decomposes an image into a pair of morphological components (for each textural characteristic), which can be associated to a smooth and a textural components. The extracted features are then used for classification with a multinomial logistic regression (MLR). The experimental results, conducted using both a hyperspectral and a synthetic aperture radar (SAR) images, reveal that the proposed scheme can lead to state-of-the-art classification accuracy.

In this letter, we explore the use of self-dual attribute profiles (SDAPs) for the classification of hyperspectral images. The hyperspectral data are reduced into a set of components by nonparametric weighted feature extraction (NWFE), and a morphological processing is then performed by the SDAPs separately on each of the extracted components. Since the spatial information extracted by SDAPs results in a high number of features, the NWFE is applied a second time in order to extract a fixed number of features, which are finally classified. The experiments are carried out on two hyperspectral images, and the support vector machines and random forest are used as classifiers. The effectiveness of SDAPs is assessed by comparing its results against those obtained by an approach based on extended APs.

The classification of an annual time series by using data from past years is investigated in this letter. Several classification schemes based on data fusion, sparse learning, and semisupervised learning are proposed to address the problem. Numerical experiments are performed on a Moderate Resolution Imaging Spectroradiometer image time series and show that while several approaches have statistically equivalent performances, a support vector machine with I1regularization leads to a better interpretation of the results due to their inherent sparsity in the temporal domain.

The application of sparse representation (SR) theory to the fusion of multispectral (MS) and panchromatic images is giving a large impulse to this topic, which is recast as a signal reconstruction problem from a reduced number of measurements. This letter presents an effective implementation of this technique, in which the application of SR is limited to the estimation of missing details that are injected in the available MS image to enhance its spatial features. We propose an algorithm exploiting the details self-similarity through the scales and compare it with classical and recent pansharpening methods, both at reduced and full resolution. Two different data sets, acquired by the WorldView-2 and IKONOS sensors, are employed for validation, achieving remarkable results in terms of spectral and spatial quality of the fused product.

Remote sensing is one of the most common ways to extract relevant information about Earth and our environment. Remote sensing acquisitions can be done by both active (synthetic aperture radar, LiDAR) and passive (optical and thermal range, multispectral and hyperspectral) devices. According to the sensor, a variety of information about the Earth's surface can be obtained. The data acquired by these sensors can provide information about the structure (optical, synthetic aperture radar), elevation (LiDAR), and material content (multispectral and hyperspectral) of the objects in the image. Once considered together their complementarity can be helpful for characterizing land use (urban analysis, precision agriculture), damage detection (e.g., in natural disasters such as floods, hurricanes, earthquakes, oil spills in seas), and give insights to potential exploitation of resources (oil fields, minerals). In addition, repeated acquisitions of a scene at different times allows one to monitor natural resources and environmental variables (vegetation phenology, snow cover), anthropological effects (urban sprawl, deforestation), climate changes (desertification, coastal erosion), among others. In this paper, we sketch the current opportunities and challenges related to the exploitation of multimodal data for Earth observation. This is done by leveraging the outcomes of the data fusion contests, organized by the IEEE Geoscience and Remote Sensing Society since 2006. We will report on the outcomes of these contests, presenting the multimodal sets of data made available to the community each year, the targeted applications, and an analysis of the submitted methods and results: How was multimodality considered and integrated in the processing chain? What were the improvements/new opportunities offered by the fusion? What were the objectives to be addressed and the reported solutions? And from this, what will be the next challenges?

Just over a decade has passed since the concept of morphological profile was defined for the analysis of remote sensing images. Since then, the morphological profile has largely proved to be a powerful tool able to model spatial information (e.g., contextual relations) of the image. However, due to the shortcomings of using the morphological profiles, many variants, extensions, and refinements of its definition have appeared stating that the morphological profile is still under continuous development. In this case, recently introduced theoretically sound attribute profiles (APs) can be considered as a generalization of the morphological profile, which is a powerful tool to model spatial information existing in the scene. Although the concept of the AP has been introduced in remote sensing only recently, an extensive literature on its use in different applications and on different types of data has appeared. To that end, the great amount of contributions in the literature that address the application of the AP to many tasks (e.g., classification, object detection, segmentation, change detection, etc.) and to different types of images (e.g., panchromatic, multispectral, and hyperspectral) proves how the AP is an effective and modern tool. The main objective of this survey paper is to recall the concept of the APs along with all its modifications and generalizations with special emphasis on remote sensing image classification and summarize the important aspects of its efficient utilization while also listing potential future works.

Pansharpening aims at fusing a multispectral and a panchromatic image, featuring the result of the processing with the spectral resolution of the former and the spatial resolution of the latter. In the last decades, many algorithms addressing this task have been presented in the literature. However, the lack of universally recognized evaluation criteria, available image data sets for benchmarking, and standardized implementations of the algorithms makes a thorough evaluation and comparison of the different pansharpening techniques difficult to achieve. In this paper, the authors attempt to fill this gap by providing a critical description and extensive comparisons of some of the main state-of-the-art pansharpening methods. In greater details, several pansharpening algorithms belonging to the component substitution or multiresolution analysis families are considered. Such techniques are evaluated through the two main protocols for the assessment of pansharpening results, i.e., based on the full- and reduced-resolution validations. Five data sets acquired by different satellites allow for a detailed comparison of the algorithms, characterization of their performances with respect to the different instruments, and consistency of the two validation procedures. In addition, the implementation of all the pansharpening techniques considered in this paper and the framework used for running the simulations, comprising the two validation procedures and the main assessment indexes, are collected in a MATLAB toolbox that is made available to the community.

Many powerful pansharpening approaches exploit the functional relation between the fusion of PANchromatic (PAN) and MultiSpectral (MS) images. To this purpose, the modulation transfer function of the MS sensor is typically used, being easily approximated as a Gaussian filter whose analytic expression is fully specified by the sensor gain at the Nyquist frequency. However, this characterization is often inadequate in practice. In this paper, we develop an algorithm for estimating the relation between PAN and MS images directly from the available data through an efficient optimization procedure. The effectiveness of the approach is validated both on a reduced scale data set generated by degrading images acquired by the IKONOS sensor and on full-scale data consisting of images collected by the QuickBird sensor. In the first case, the proposed method achieves performances very similar to that of the algorithm that relies upon the full knowledge of the degrading filter. In the second, it is shown to outperform several very credited state-of-the-art approaches for the extraction of the details used in the current literature.

Classification is one of the most important techniques to the analysis of hyperspectral remote sensing images. Nonetheless, there are many challenging problems arising in this task. Two common issues are the curse of dimensionality and the spatial information modeling. In this paper, we present a new general framework to train series of effective classifiers with spatial information for classifying hyperspectral data. The proposed framework is based on the two key observations: 1) the curse of dimensionality and the high feature-to-instance ratio can be alleviated by using random subspace (RS) ensembles; and 2) the spatial-contextual information is modeled by the extended multiattribute profiles (EMAPs). Two fast learning algorithms, i.e., decision tree (DT) and extreme learning machine (ELM), are selected as the base classifiers. Six RS ensemble methods, namely, RS with DT, random forest (RF), rotation forest, rotation RF (RoRF), RS with ELM (RSELM), and rotation subspace with ELM (RoELM), are constructed by the multiple base learners. Experimental results on both simulated and real hyperspectral data verify the effectiveness of the RS ensemble methods for the classification of both spectral and spatial information (EMAPs). On the University of Pavia Reflective Optics Spectrographic Imaging System image, our proposed approaches, i.e., both RSELM and RoELM with EMAPs, achieve the state-of-the-art performances, which demonstrates the advantage of the proposed methods. The key parameters in RS ensembles and the computational complexity are also investigated in this paper.

Path operators are powerful tools for the enhancement of thin and elongated objects in an image. In order to cope with noisy acquisition a variant of the path operators was recently proposed. However, both approaches cannot properly handle thin objects with tortuous shapes since strong variations of an object curvature produce disconnections in the paths. In order to address this issue, we propose a novel operator able to properly handle paths in tortuous shapes. It relies on the coupling of attribute filters based on the geodesic tortuosity and conventional path operators. Analogously to the complete version of the path operators, by allowing disconnections within paths it is possible also to define a path operator that is both robust and selective. The effectiveness of the proposed operators in filtering thin and tortuous image objects is proved on a 2D and 3D biomedical image.

Pansharpening is a successful application of data fusion to remotely sensed data. It aims at obtaining a detailed representation of an Earth's zone both in terms of spatial and spectral resolution. This is done through the fusion of a panchromatic and a multispectral image (having complementary spatial and spectral resolutions) that are acquired simultaneously by several optical satellites. The result of the fusion is commonly achieved by introducing the spatial details, modulated opportunely by gains, in the multispectral one. The injection gains can be estimated globally over the image, or locally, thus obtaining spatially variant values. The latter approach has been proven to achieve better results and it is based on windowing the analyzed image in squared blocks. In this paper we propose a more elaborated concept of locality, as it is based on an opportune segmentation of the target scene. In greater details, we propose to estimate the local injection gains on regions composed of pixel with similar spectral characteristic, as defined by a segmentation. Such local approach is compared to the global one and to the conventional local estimation based on overlapping and non-overlapping blocks. The performances have been assessed by using three real datasets, the first acquired by WorldView-2 and the other two by Pléiades. The analysis evidences the appreciable improvements of the performances with respect to classical schemes.

In this paper we compare features obtained by different filtering strategies for morphological attribute filters by considering non-increasing attributes. The Attribute profiles (APs) and Self Dual Attribute Profiles (SDAPs) are obtained by sequentially applying attribute filters on tree-based image representations, such as Min- or Max-trees and Inclusion tree, respectively. This work aims to study the effects of using the filtering rules max, min, direct and subtractive, when considering the non-increasing attributes moment of inertia and standard deviation. A very high spatial resolution data set is used in the experiments, and the extracted information obtained by the profiles is analyzed. This is done by studying the effects on the classification accuracy by using the profiles as additional input features to a Random Forest classifier.

The automatic detection of hedgerows in very high resolution remote sensing images is addressed in this paper. In particular, the use of advanced morphological filters, such as path operators, is proposed. Conventional path openings have been already proposed in the literature to discriminate between forest objects and hedge objects in very high resolution optical images. They have shown greater flexibility with respect to geodesic openings. However, path operators are sensitive to noise and in practical situations they are likely to produce missed detections. In particular, path operators are unable to extract long hedgerows as a single object. In order to tackle this limitation, robust path openings are investigated in this work. In the experimental results, robust path opening shows superior performances for the detection of hedgerows.

The snow coverage area (SCA) is one of the most important parameters for cryospheric studies. The use of remote sensing imagery can complement field measurements by providing means to derive SCA with a high temporal frequency and covering large areas. Images acquired by the Moderate Resolution Imaging Spectroradiometer (MODIS) are perhaps the most widely used data to retrieve SCA maps. Some MODIS derived algorithms are available for subpixel SCA estimation, as MODSCAG and MODImLab. Both algorithms make use of spectral unmixing techniques using a fixed set of snow, rocks and other materials spectra (endmembers). We aim to improve the performance of a modified version of MODIm-Lab algorithm by exploring advanced spectral unmixing techniques. Furthermore, we make use of endmember induction algorithms to obtain the endmembers from the data itself instead of using a fixed spectral library. We validate the proposed approach on a case study in the mountainous region of the Alps.

The design of a pansharpening algorithm for enriching a MultiSpectral image with the spatial details of a Panchromatic image should preserve the characteristics of the original dataset. A widely employed quality check consists in verifying the consistency of the fused product, namely the similarity of the original image and a reduced resolution version of the sharpened product. We propose to improve this feature by applying an Iterative Back-Projection algorithm after the fusion procedure. The approach is validated on two datasets, acquired by the Ikonos and WorldView-2 sensors, showing remarkable improvements, especially in conjunction with Component Substitution pansharpening methods.

In this paper state-of-the-art and advanced methods for multispectral pansharpening are reviewed and evaluated on two very high resolution datasets acquired by IKONOS-2 (four bands) and WorldView-2 (eight bands). The experimental analysis allows us to highlight the performances of the two main pansharpening approaches (i.e. component substitution and multiresolution analysis).

Images with high spatial and spectral resolutions are desirable for remote sensing applications. Unfortunately, due to sensor physical constraints, this result cannot be obtained by a single sensor. To overcome these limitations, a great number of data fusion approaches have been developed in the last years. The fusion of panchromatic and multispectral images, also known as Pansharpening, is capturing a lot of attention in the literature. In this paper, we extend and analyze the use of some classical pansharpening techniques, belonging to the MultiResolution Analysis and Component Substitution families, for fusing hyperspectral data instead of multispectral ones. The experimental results, conducted on two real datasets acquired by the Hyperion/ALI and CHRIS-Proba/QuickBird sensors, point out the greater suitability of the algorithms into the MRA class thanks to a better spectral consistency of the final products, which is a desirable feature when the number of bands to fuse increases.

Over the last few years, several new strategies have been proposed for spectral-spatial classification of remotely sensed image data, for cases when high spatial and spectral resolutions are available. In this letter, we focus on the possibility of performing advanced spectral-spatial classification of remote sensing images with limited spectral resolution (often called multispectral). A new strategy is proposed, where the spectral dimensionality of the multispectral data is first expanded by using nonlinear feature extraction with kernel methods such as kernel principal component analysis. Then, extended multiattribute profiles (EMAPs), built on the expanded set of spectral features, are used to include spatial information. This strategy allows us to first decompose different spectral clusters into different spectral features and further improve the spatial discrimination. The resulting EMAPs are used for classification using advanced classifiers such as support vector machines and random forests. We test our proposed methodology with different multispectral data sets obtaining state-of-the-art classification results.

The pansharpening process has the purpose of building a high-resolution multispectral image by fusing low spatial resolution multispectral and high-resolution panchromatic observations. A very credited method to pursue this goal relies upon the injection of details extracted from the panchromatic image into an upsampled version of the low-resolution multispectral image. In this letter, we compare two different injection methodologies and motivate the superiority of contrast-based methods both by physical consideration and by numerical tests carried out on remotely sensed data acquired by IKONOS and Quickbird sensors.

In recent years, sparse representations have been widely studied in the context of remote sensing image analysis. In this paper, we propose to exploit sparse representations of morphological attribute profiles for remotely sensed image classification. Specifically, we use extended multiattribute profiles (EMAPs) to integrate the spatial and spectral information contained in the data. EMAPs provide a multilevel characterization of an image created by the sequential application of morphological attribute filters that can be used to model different kinds of structural information. Although the EMAPs' feature vectors may have high dimensionality, they lie in class-dependent low-dimensional subpaces or submanifolds. In this paper, we use the sparse representation classification framework to exploit this characteristic of the EMAPs. In short, by gathering representative samples of the low-dimensional class-dependent structures, any given sample may by sparsely represented, and thus classified, with respect to the gathered samples. Our experiments reveal that the proposed approach exploits the inherent low-dimensional structure of the EMAPs to provide state-of-the-art classification results for different multi/hyperspectral data sets.

Including spatial information is a key step for successful remote sensing image classification. In particular, when dealing with high spatial resolution, if local variability is strongly reduced by spatial filtering, the classification performance results are boosted. In this paper, we consider the triple objective of designing a spatial/spectral classifier, which is compact (uses as few features as possible), discriminative (enhances class separation), and robust (works well in small sample situations). We achieve this triple objective by discovering the relevant features in the (possibly infinite) space of spatial filters by optimizing a margin-maximization criterion. Instead of imposing a filter bank with predefined filter types and parameters, we let the model figure out which set of filters is optimal for class separation. To do so, we randomly generate spatial filter banks and use an active-set criterion to rank the candidate features according to their benefits to margin maximization (and, thus, to generalization) if added to the model. Experiments on multispectral very high spatial resolution (VHR) and hyperspectral VHR data show that the proposed algorithm, which is sparse and linear, finds discriminative features and achieves at least the same performances as models using a large filter bank defined in advance by prior knowledge.

The binary partition tree (BPT) is a hierarchical region-based representation of an image in a tree structure. The BPT allows users to explore the image at different segmentation scales. Often, the tree is pruned to get a more compact representation and so the remaining nodes conform an optimal partition for some given task. Here, we propose a novel BPT construction approach and pruning strategy for hyperspectral images based on spectral unmixing concepts. Linear spectral unmixing consists of finding the spectral signatures of the materials present in the image (endmembers) and their fractional abundances within each pixel. The proposed methodology exploits the local unmixing of the regions to find the partition achieving a global minimum reconstruction error. Results are presented on real hyperspectral data sets with different contexts and resolutions.

The Binary Partition Tree (BPT) is a hierarchical region-based representation of an image in a tree structure. BPT allows users to explore the image at different segmentation scales, from fine partitions close to the leaves to coarser partitions close to the root. Often, the tree is pruned so the leaves of the resulting pruned tree conform an optimal partition given some optimality criterion. Here, we propose a novel BPT construction approach and pruning strategy for hyperspectral images based on spectral unmixing concepts. The proposed methodology exploits the local unmixing of the regions to find the partition achieving a global minimum reconstruction error. We successfully tested the proposed approach on the well-known Cuprite hyperspectral image collected by NASA Jet Propulsion Laboratory's Airborne Visible/Infrared Imaging Spectrometer (AVIRIS). This scene is considered as a standard benchmark to validate spectral unmixing algorithms.

Inclusion of spatial information is known to be beneficial to the classification of hyperspectral images. However, given the high dimensionality of the data, it is difficult to know before hand which are the bands to filter or what are the filters to be applied. In this paper, we propose an active set algorithm based on a l1 support vector machine that explores the (possibily infinite) space of spatial filters and retrieves automatically the filters that maximize class separation. Experiments on hyperspectral imagery confirms the power of the method, that reaches state of the art performance with small feature sets generated automatically and without prior knowledge.

Remote sensors capture large scenes that are conventionally split in smaller patches before being stored and analyzed. Traditionally, this has been done by dividing the scene in rectangular windows. Such windowing methodology could provoke the separation of spectrally homogeneous areas or objects of interest into two or more patches. This is due to the presence of objects of interest in correspondence to windows' borders, or because the fixed size of the windows does not adapt well to the scale of the objects. To alleviate this issue, the windows can be arranged in an overlapping way, incurring in some data redundancy storage. Recently, tree representations have been used as an alternative to windowing in order to structure and store large amounts of remote sensing data. In this work we explore the benefits of using Binary Partition Trees (BPT) instead of windowing to store hyperspectral large scenes. We are particularly interested in storing the information resulting of local spectral unmixing processes running over a large real hyperspectral scene. We show that under similar conditions BPT allows a better storage of the unmixing information in terms of reconstruction error.

A new approach to change detection in very high resolution remote sensing images based on morphological attribute profiles (APs) is presented. A multiresolution contextual transformation performed by APs allows the extraction of geometrical features related to the structures within the scene at different scales. The temporal changes are detected by comparing the geometrical features extracted from the image of each date. The experiments performed on panchromatic QuickBird images related to an urban area show the effectiveness of the proposed technique in detecting changes on the basis of the spatial morphology by preserving geometrical detail.

Extended attribute profiles, which are based on attribute filters, have recently been presented as efficient tools for spectral-spatial classification of remote sensing images. However, construction of these profiles usually requires manual selection of parameters for the corresponding attribute filters. In this letter, we present a technique to automatically build the extended attribute profiles with the standard deviation attribute based on the statistics of the samples belonging to the classes of interest. The methodology is tested on two widely used hyperspectral images and the results are found to be highly accurate.

Morphological and attribute profiles have been proven to be effective tools to fuse spectral and spatial information for classification of remote sensing data. A wide range of filters (i.e., number of levels in the profiles) is usually necessary in order to properly model the spatial information in a remote sensing scene. A dense sampling of the values of the parameters of the filters generates profiles that have both a very large dimensionality (leading to the Hughes phenomenon in classification) and a high redundancy. In this paper, a novel iterative technique based on genetic algorithms (GAs) is proposed to automatically optimize the selection of the optimal features from the profiles. The selection of the filtered images that compose the profile is performed by dividing them into three classes corresponding to high, medium, and low importance. We propose to measure the importance (modeled in terms of discriminative power in the classification task) using a random forest classifier, which provides a rank for each feature with its model. Only the set of images associated with the highest importance is selected, i.e., preserved for classification. The proposed technique is applied to the features labeled with medium importance, whereas the images with the lowest importance are removed from the profile. This method is employed to classify three hyperspectral data sets achieving significantly high classification accuracy values. A parallel computing implementation has been developed in order to significantly reduce the time required for the run of the GAs.

This paper presents a classification method for hyperspectral images utilizing Differential Morphological Profiles (DMPs) which permit to include in the analysis spatial information since they can provide an estimate of the size and contrast characteristics of the structures in an image. Due to the wide variety of objects present in a scene, the pixels belonging to the same semantic structure may not have homogeneous spatial and spectral features. In addition, instead of a single peak (which can be related to a measure of the scale), multiple local maxima and multiple responses are usually observed in the DMP. In order to handle such intra-class variability, class-specific weighting functions are employed in order to differently modulate the DMP values according to the different characteristics of the land cover types. In such way, it is possible to differentiate the behaviors of the DMP for each pixel in the image according to its semantic, providing an increase of the separability of the classes. At first, a DMP computed with opening by reconstruction (DMPO) and one with closing by reconstruction (DMPC) are derived on each of the first principle components extracted from the hyperspectral image. Then, both profiles are weighted by each class-specific weighting function and concatenated in a single data structure. The constructed feature vectors are considered by a random forest classifier.

Digital Elevation Models (DEMs) are a valuable resource in geoscience and remote sensing. However, not all DEMs demonstrate the same level of accuracy, resolution and precision. Moreover, in some regions of the globe coverage is either sparse or non-existent, for example at the poles where ice also hampers accurate readings. In this paper, we propose to use collective archives of geo-located landscape photos acquired at ground-level to correct, validate and possibly reconstruct DEMs.

Augmented Reality (AR) offers a means to inject virtual information into real scenes. In the past few years, AR has been receiving greater attention thanks to considerable advancements in the hardware of consumer-level portable devices. In this paper, we illustrate how mobile AR can be exploited to intuitively visualize, and moreover generate new, geo-data. We explore these two concepts through i) the visualization and interaction modality of geo-data as AR layers and ii) the exploitation of mobile devices as opportunistic sensors for generating information relating to a user's immediate surroundings.

Dealing with a high number of features belonging to different types of data such as Hyperspectral image and Morphological Attribute Profiles (MAPs) might lead to a poor predictive performance of the classifier and hence low final accuracies of classification. This is due to the Hughes effect that consistently decreases the power of prediction of the classifier, in case of a limited and fixed number of training samples. In order to reduce the number of features and only keeping those which are more informative, a novel supervised feature selection technique based on GAs and the measure of the relevance of the features is presented in this work. Moreover, the effectiveness of the proposed technique was demonstrated by experimenting on an optical remote sensed dataset.

With respect to satellite and aerial RGB images, ground-based acquisitions can provide a more detailed representation of a natural landscape, especially for steep slopes. Moreover, for some applications the generation of a new `virtual' view-point can provide a valuable visualization tool. Using such a technique, the visual appearance of a landscape (generated from one or several geo-registered images) can be seen from a new, specified vantage point. In this paper, we propose a method of generating a visual appearance model and subsequent virtual views, through a direct re-projection of visual content from source geo-registered images.

In this paper a general approach based on morphological connected filters for the spatial simplification of very high resolution remote sensing images is introduced. In greater detail, the proposed approach is made up of two steps: i) the selection of the parameters defining the connected filters driven by the information available on the scene and on the specific application; and ii) the application of the tuned filter to the input image. This work aims at: i) explicitly delineating the characteristic of an approach for the spatial simplification of images based on connected filters; ii) defining a general architecture suitable for the analysis in different scenarios modeling common different operative conditions; iii) giving guidelines for the automation of the simplification process according to different operational settings; iv) qualitatively evaluating the application of the proposed approach on a real data set in different scenarios.

Extended Morphological Attribute Profiles (EAPs) are extension of Extended Morphological Profiles (EMPs). They are based on the more general Morphological Attribute Profiles (APs) rather than the conventional Morphological Profiles (MPs). EAPs are computed on few of the first principle components (PCs) extracted from the multi-/hyper-spectral data. In this paper, we propose to compute EAPs on features derived from supervised feature extraction techniques such as discriminant analysis feature extraction (DAFE), decision boundary feature extraction (DBFE) and non-parametric weighted feature extraction (NWFE)) instead of using unsupervised principal component analysis (PCA).

In this letter, a technique based on independent component analysis (ICA) and extended morphological attribute profiles (EAPs) is presented for the classification of hyperspectral images. The ICA maps the data into a subspace in which the components are as independent as possible. APs, which are extracted by using several attributes, are applied to each image associated with an extracted independent component, leading to a set of extended EAPs. Two approaches are presented for including the computed profiles in the analysis. The features extracted by the morphological processing are then classified with an SVM. The experiments carried out on two hyperspectral images proved the effectiveness of the proposed technique.

In this paper we investigate the combined use of morphological attribute filters and feature extraction techniques for the classification of a high resolution hyperspectral image. In greater detail, we propose to model the spatial information with Extended Attribute Profiles computed on the hyperspectral data and to reduce the high dimensionality of the morphological features computed (which show a high degree of redundancy) with feature extraction techniques. The features extracted are analyzed by two classifiers. The experimental analysis was carried out on a high resolution hyperspectral image acquired by the airborne sensor ROSIS-03 on the University of Pavia, Italy. The obtained results compared to those obtained without feature reduction proved the importance of the application of a stage of feature extraction in the process.

Morphological attribute profiles (APs) are defined as a generalization of the recently proposed morphological profiles (MPs). APs provide a multilevel characterization of an image created by the sequential application of morphological attribute filters that can be used to model different kinds of the structural information. According to the type of the attributes considered in the morphological attribute transformation, different parametric features can be modeled. The generation of APs, thanks to an efficient implementation, strongly reduces the computational load required for the computation of conventional MPs. Moreover, the characterization of the image with different attributes leads to a more complete description of the scene and to a more accurate modeling of the spatial information than with the use of conventional morphological filters based on a predefined structuring element. Here, the features extracted by the proposed operators were used for the classification of two very high resolution panchromatic images acquired by Quickbird on the city of Trento, Italy. The experimental analysis proved the usefulness of APs in modeling the spatial information present in the images. The classification maps obtained by considering different APs result in a better description of the scene (both in terms of thematic and geometric accuracy) than those obtained with an MP.

This paper proposes the use of morphological attribute profiles as an effective alternative to the conventional morphological operators based on the geodesic reconstruction for modeling the spatial information in very high resolution images. Attribute profiles, used in multilevel approaches, result particularly effective in terms of computational complexity and capabilities in characterizing the objects in the image. In addition they are more flexible than operators by reconstruction, thanks to the definition of possible different attributes. Experimental results obtained on a Quickbird panchromatic very high resolution image proved the effectiveness of the presented attribute filters and pointed out their main properties.

An unsupervised technique for change detection (CD) in very high geometrical resolution images is proposed, which is based on the use of morphological filters. This technique integrates the nonlinear and adaptive properties of the morphological filters with a change vector analysis (CVA) procedure. Different morphological operators are analyzed and compared with respect to the CD problem. Alternating sequential filters by reconstruction proved to be the most effective, permitting the preservation of the geometrical information of the structures in the scene while filtering the homogeneous areas. Experimental results confirm the effectiveness of the proposed technique. It increases the accuracy of the CD process as compared with the standard CVA approach.