The Sentinel-2 (S2) satellite constellation provides images at three different spatial resolutions and model based superresolution methods have proved useful for sharpening them to their maximum resolution. Algorithm unrolling is a way of building efficient, interpretable neural networks by reimplementing traditional algorithms in a neural network context. In this paper, an unrolled model based method to superresolve S2 images is proposed and unsupervised single image training is performed using reduced scale data. The method is evaluated using both real and simulated data.

This paper proposes a new loss function to train a convolutional neural network (CNN) for multispectral and hyper-spectral (MS-HS) image fusion. The loss function is based on the relative dimensionless global error synthesis (ER-GAS), where we exchange the mean squared error (MSE) for its unbiased estimate using Stein’s risk unbiased estimate (SURE). The loss function has a good balance between the spectral and spatial information implied by the weighted MSE, therefore it does not need a parameter to balance the spectral and spatial terms as in MSE loss function, and it also converges faster than the MSE one. Additionally, the loss function enables unsupervised training and avoids overfit-ting, since it is derived by using SURE. Experimental results show that the proposed method yields good results and outperforms the competitive methods. Codes are available at https://github.com/hvn2/SURE-ERGAS

Multispectral remote sensing images are often have band-dependent image resolution due to cost and technical limitations. To address this, we developed a method that sharpens low-resolution (LR) images using high-resolution (HR) images. In this paper, we propose a novel unsupervised deep learning (DL) approach that involves unrolling an iterative algorithm into a deep neural network and training it using a loss function based on Stein’s risk unbiased estimate (SURE) to sharpen the LR bands (20 and 60 m) of Sentinel-2 (S2) to their highest resolution (10 m). This approach views traditional optimization model-based methods through a DL framework, improving interpretability and clarifying connections between the two approaches. Results from both simulated and real S2 datasets demonstrate that the proposed method outperforms competitive methods and produces high-quality images for the 20 m and 60 m bands. The codes are available at: https://github.com/hvn2/S2-Unrolling.

Recent advances in deep learning (DL) reveal that the structure of a convolutional neural network (CNN) is a good image prior (called deep image prior (DIP)), bridging the model-based and DL-based methods in image restoration. However, optimizing a DIP-based CNN is prone to over-fitting leading to a poorly reconstructed image. This paper derives a loss function based on Stein's unbiased risk estimate (SURE) for unsupervised training of a DIP-based CNN applied to the hyperspectral image (HSI) super-resolution. The SURE loss function is an unbiased estimate of the mean-square-error (MSE) between the clean low-resolution image and the low-resolution estimated image, which relies only on the observed low-resolution image. Experimental results on HSI show that the proposed method not only improves the performance, but also avoids overfitting. Codes are available at https://github.com/hvn2/SURE-MS-HS

Spectral and spatial correlation in hyperspectral images (HSIs) can be exploited in HSI processing because it directly induces a sparse and low-rank prior via linear transformations. Researchers have used the sparse and low-rank prior as an image prior for HSI restoration, such as denoising, deblurring, and super-resolution. This paper proposes a HSI denoising method that incorporates a sparse and low-rank prior with a deep image prior (DIP). The sparse and low-rank prior is obtained using the 2-dimensional discrete wavelet transform (2-D DWT), and singular value decomposition (SVD), while the DIP is provided by the structure of a convolutional neural network (CNN). The combination of a sparse and low-rank prior with a DIP views the CNN-based denoising method similar to a model-based method, inheriting the advantages of both model-based and CNN-based methods. Experimental results with simulated and real HSI datasets show that the proposed method outperforms the conventional sparse and low-rank based methods in both quantitative and qualitative performance. Codes are available at https://github.com/hvn2/DIP-SLR

Spectral variability in hyperspectral images (HSIs) has received lot of attention over the last years, especially in the field of hyperspectral unmixing (HU) where it is a major issue. In this letter, we propose a method utilizing a variational autoencoder (VAE) for creating synthetic HSIs having controllable degree of spectral variability from existing HSIs with established ground-truth abundance maps and endmembers. Such synthetic datasets can be useful for developing HU methods that can handle spectral variability in HSIs. We investigate how the variability in the synthetic images differs from the original images and perform blind unmixing experiments using generated datasets to illustrate the effect of increasing variability. Code for method is available athttps://github.com/burknipalsson/vae_synthetic_hsi.

This letter proposes a semi-supervised mixtures of factor analyzers (S2MFA) feature extraction (FE) method for hyperspectral image (HSI). S2MFA uses a Gaussian mixture model to segment the image to different regions, each region follows a Gaussian distribution and contains labeled and unlabeled samples. The method uses a factor analyzer to get a factor-loading matrix to preserve the local spatial information using the labeled and unlabeled samples. It simultaneously improves the class discrimination of the data using the labeled samples and also transforms the original image to an optimal low-dimensional subspace to achieve dimensionality reduction. The performance of the S2MFA FE method is evaluated by classification of two real HSIs and compared to different kinds of statistic unsupervised, supervised, and semi-supervised FE methods.

Image fusion is utilized in remote sensing due to the limitation of the imaging sensor and the high cost of simultaneously acquiring high spatial and spectral resolution images. Optical remote sensing imaging systems usually provide images of high spatial resolution but low spectral resolution and vice versa. Therefore, fusing those images to obtain a fused image having both high spectral and spatial resolution is desirable in many applications. This paper proposes a fusion framework using an unsupervised convolutional neural network (CNN) and Stein’s unbiased risk estimate (SURE). We derive a new loss function for a CNN that incorporates back-projection mean-squared error with SURE to estimate the projected mean-square-error (MSE) between the fused image and the ground truth. The main motivation is that training a CNN with this SURE loss function is unsupervised and avoids overfitting. Experimental results for two fusion examples, multispectral and hyperspectral (MS-HS) image fusion, and multispectral and multispectral (MS-MS) image fusion, show that the proposed method yields high quality fused images and outperforms the competitive methods. Codes are be available at https://github.com/hvn2/Deep-SURE-Fusion.

This article proposes a denoising method based on sparse spectral–spatial and low-rank representations (SSSLRR) using the 3-D orthogonal transform (3-DOT). SSSLRR can be effectively used to remove the Gaussian and mixed noise. SSSLRR uses 3-DOT to decompose noisy HSI to sparse transform coefficients. The 3-D discrete orthogonal wavelet transform (3-D DWT) is a representative 3-DOT suitable for denoising since it concentrates on the signal in few transform coefficients, and the 3-D discrete orthogonal cosine transform (3-D DCT) is another example. An SSSLRR using 3-D DWT will be called SSSLRR-DWT. SSSLRR-DWT is an iterative algorithm based on the alternating direction method of multipliers (ADMM) that uses sparse and nuclear norm penalties. We use an ablation study to show the effectiveness of the penalties we employ in the method. Both simulated and real hyperspectral datasets demonstrate that SSSLRR outperforms other comparative methods in quantitative and visual assessments to remove the Gaussian and mixed noise.

In recent years numerous model-based methods for super-resolution of Sentinel-2 (S2) multispectral images have been suggested. Super-resolution aims to enhance the resolution of a captured image by upscaling and enhancing the details. The performance of model-based methods relies on carefully selecting regularizers and tuning parameters. This paper investigates whether using Wald's protocol, i.e., selecting tuning parameters at reduced-resolution, translates to a good performance at a full-scale. To investigate this, we use the recently proposed S2Sharp method and show that selecting its tuning parameters using Wald's protocol improves its performance.

This paper proposes a novel method for sharpening the 20 m bands of the multispectral images acquired by the Sentinel-2 (S2) constellation. We formulate the S2 sharpening as an inverse problem and solve it using an unsupervised convolutional neural network (CNN), called S2UCNN. The proposed method extends the deep image prior provided by a CNN structure with S2 domain knowledge. We incorporate a modulation transfer function-based degradation model as a network layer. We add the 10 m bands to both the network input and output to take advantage of the multitask learning. Experimental results with a real S2 dataset show that the proposed method outperforms the competitive methods on reduced-resolution data and gives very high quality sharpened image on full-resolution data.

Sentinel-2 (S2) is a constellation of two satellites that frequently acquire optical imagery over land and coastal waters. The S2 sensors have three spatial resolutions: 10, 20, and 60 m. Many remote sensing applications require the spatial resolution to be at the highest resolution, i.e., 10 m for S2. To address this demand, researchers have proposed various methods that exploit the spectral and spatial correlation in multispectral data to sharpen the S2 bands to 10 m. In this paper, we fuse S2 data with high-resolution aerial RGB images. A method called S2Sharp is modified to include the red, green, and blue bands of the aerial image and sharpen S2 data to the resolution of the RGB image. The method, termed S2PF, is evaluated using an S2 image and aerial photographs of Reykjavik, Iceland.

This paper presents a wavelet-based block low-rank representations (WBBLRR) denoising method for hyperspectral images (HSIs). WBBLRR uses 3-D wavelet transformation to decompose HSI into different blocks, where each block utilizes a low-rank representations model to obtain the denoised block, and then uses inverse 3-D wavelet transformation for all the denoised blocks to obtain the denoised HSI. The proposed method is evaluated by using both simulated and real hyperspectral datasets.

This paper presents a non-local means low-rank approximation (NLMLRA) denoising method for hyperspectral images (HSIs). NLMLRA uses a Slanted Butterworth function to construct a low-rank approximation for non-local means (NLM) operator and is efficiently implemented based on Chebyshev polynomials. The proposed method is evaluated by using both simulated and real hyperspectral datasets.

Hyperspectral images (HSIs) are useful for many remote sensing applications. However, they are usually affected by noise that degrades the HSIs quality. Therefore, HSI denoising is important to improve the performance of subsequent HSI processing and analysis. In this article, we propose an HSI denoising method called Stein's unbiased risk estimate-convolutional neural network (SURE-CNN). The method is based on an unsupervised CNN and SURE. The main difference of SURE-CNN from existing supervised learning methods is that the SURE-based loss function can be computed only from noisy data. Since SURE is an unbiased estimate of the mean squared error (MSE) of an estimator, training a CNN using the SURE loss can yield similar results as using the MSE with ground truth in supervised learning. Also, a subspace version of SURE-CNN is proposed to reduce the running time. Extensive experimental results with both simulated and real data sets show that the SURE-CNN method outperforms the competitive methods in both objective and subjective assessments.

Blind hyperspectral unmixing is the process of expressing the measured spectrum of a pixel as a combination of a set of spectral signatures called endmembers and simultaneously determining their fractional abundances in the pixel. Most unmixing methods are strictly spectral and do not exploit the spatial structure of hyperspectral images (HSIs). In this article, we present a new spectral-spatial linear mixture model and an associated estimation method based on a convolutional neural network autoencoder unmixing (CNNAEU). The CNNAEU technique exploits the spatial and the spectral structure of HSIs both for endmember and abundance map estimation. As it works directly with patches of HSIs and does not use any pooling or upsampling layers, the spatial structure is preserved throughout and abundance maps are obtained as feature maps of a hidden convolutional layer. We compared the CNNAEU method to four conventional and three deep learning state-of-the-art unmixing methods using four real HSIs. Experimental results show that the proposed CNNAEU technique performs particularly well and consistently when it comes to endmembers' extraction and outperforms all the comparison methods.

Hyperspectral images (HSI) are composed of hundreds of spectral bands, covering a broad range of the electromagnetic spectrum. However, images can only be visualized using three spectral channels for red, green, and blue (RGB) colors. Generating realistic RGB images using HSI is seldom the main focus of remote sensing researchers, and is therefore sometimes lacking. In this paper, we present an algorithm which creates realistic color images of HSI, using standardized methods. Research, conducted on the human perception of color in the 1920s culminated in the CIE 1931 XYZ color space. The algorithm maps every spectral band in the visible spectrum to the XYZ color space, using D65 as the reference illuminant, and then maps the XYZ to the sRGB (standard Red Green Blue) color space. The image is gamma-corrected and finally thresholded to improve contrast. The method was validated using two HSIs, creating realistic color images.

This paper addresses the hyperspectral image (HSI) denoising problem by using Stein's unbiased risk estimate (SURE) based convolutional neural network (CNN). Conventional deep learning denoising approaches often use supervised methods that minimize a mean-squared error (MSE) by training on noisy-clean image pairs. In contrast, our proposed CNN-based denoiser is unsupervised and only makes use of noisy images. The method uses SURE, which is an unbiased estimator of the MSE, that does not require any information about the clean image. Therefore minimization of the SURE loss function can accurately estimate the clean image only from noisy observation. Experimental results on both simulated and real hyperspectral datasets show that our proposed method outperforms competitive HSI denoising methods.

Sentinel-2 (S2) satellite constellations can provide multispectral images of 10 m, 20 m, and 60 m resolution for visible, near-infrared (NIR) and short-wave infrared (SWIR) in the electromagnetic spectrum. In this paper, we present a sharpening method based on a symmetric skipped connection convolutional neural network, called SSC-CNN, to sharpen 20 m bands using 10 m bands. The main advantage of SSC-CNN architecture is that it brings the features of the input branch to the output, thus improving convergence without using too many deep layers. The proposed method uses the reduced-scale combination of 10 m bands and 20 m bands, and the observed 20 m bands as the training pairs. The experimental results using two Sentinel-2 datasets show that our method outperforms competitive methods in quantitative metrics and visualization.

This paper presents a local spatial-spectral correlation based mixtures of factor analyzers (LSSC-MFA) denoising method for hyperspectral image (HSI). HSIs are usually degraded by different noise types such as missing lines (ML), missing pixels (MP), salt and pepper noise (SP), and Gaussian noise. The proposed method, hierarchically, removes the mixed noise. Firstly, we develop a novel local spatial-spectral correlation (LSSC) method to remove the ML noise. Then LSSC-MFA uses the mixtures of factor analyzers (MFA) method to remove the MP, SP, and Gaussian noises. The performance of the proposed method has been validated using both real and simulated HSI datasets. Results on the simulated datasets confirm considerable improvements in terms of peak signal-to-noise ratio (PSNR) compared to the state-of-the-art denoising methods used in experiments. In addition, visual improvements can be observed in the case of real dataset experiments.

This paper presents two hyperspectral image (HSI) denoising methods, mixtures of factor analyzers (MFA) and wavelet-based MFA (WMFA). MFA uses a Gaussian mixture model to segment the original HSI into different parts, where each part follows Gaussian distribution and then utilizes a factor analyzer to get a low-rank factor loading matrix, and finally uses the inverse transformation of the matrix to get the denoised hyperspectral dataset. WMFA uses the MFA in the wavelet domain to remove the noise in HSI. The proposed methods are evaluated by using both simulated and real hyperspectral datasets.

Observation of the Earth using satellites mounted with optical sensors is an important application of remote sensing. Owing to physical constraints, multispectral (MS) sensors acquire images of lower spatial resolution than a single-band panchromatic (PAN) sensor that acquires images of the same scene. Pansharpening fuses the MS and PAN images to obtain an MS image with the same spatial resolution as the PAN image. In this letter, we propose to expand a method, initially developed for Sentinel-2 single-sensor sharpening, for pansharpening. The expanded method is based on solving a non-convex MS acquisition model using optimization methods based on cyclic decent and manifold optimization. The tuning parameters of the method are chosen using Bayesian optimization with reduced-scale evaluation. The proposed method is compared with a number of established pansharpening methods and is validated using both synthetic and real data sets.

This paper introduces a novel deep learning based method for blind hyperspectral unmixing. The technique utilizes multi-task learning (MTL) through multiple parallel autoencoders to simultaneously unmix a neighborhood of pixels at a time. The sharing of hidden representations between autoencoders enables the method to take advantage of spatial correlations in the hyperspectral image (HSI). The method is evaluated using real HSI data and compared to three state-of-the-art techniques. The proposed method outperforms all the comparison methods in the experiments.

The fusion of a low spatial resolution multispectral image and a high spatial resolution panchromatic image, i.e., pan-sharpening is an important technique in remote sensing where high resolution imagery is needed. Two of the largest families of such methods are the component substitution (CS) and multi-resolution analysis (MRA) methods. These families of methods can be described by general detail injection schemes which are closely related. In this paper, we propose pansharpening methods which are based on directly implementing these schemes using a convolutional neural network (CNN) such that the mean squared error between the down-sampled fused image and the observed multispectral image is minimized. Using a simulated Pleiades dataset we demonstrate that the proposed approach gives excellent results when compared to other state-of-the-art CS, MRA and CNN methods.

In this paper, we present a deep learning based method for blind hyperspectral unmixing in the form of a fully convolutional autoencoder. The technique is the first to fully utilize the spatial structure of hyperspectral images (HSIs) for both endmember and abundance map estimation. The framework has many advantages over older methods as it works directly with patches of HSIs' and thus preserves the spatial structure while abundance maps arise naturally as feature maps of a hidden convolutional layer. We evaluate the proposed method using a real HSI and compare it to three state-of-the-art methods. The proposed method outperforms all the comparison methods in the experiments.

Blind hyperspectral unmixing is the process of decomposing hyperspectral images (HSIs) into pure material spectra (endmembers) and abundances. In this paper, we examine scaling the pixels of the HSI inversely proportional to their ℓ2norm, controlled with a tuning parameter. We promote sparse abundances using an ℓqpenalty and softly enforce the abundance sum constraint using matrix augmentation. The minimization problem is solved using a variant of sparse nonnegative matrix factorization (NMF) and all tuning parameters are selected using Bayesian optimization. The proposed method is evaluated using two real hyperspectral images.

This paper presents four dimensionality reduction methods, supervised mixtures of factor analyzers (SMFA), semi-supervised mixtures of factor analyzers (S2MFA), supervised deep mixtures of factor analyzers (SDMFA) and semi-supervised deep mixtures of factor analyzers (S2DMFA), for hyperspectral image (HSI) classification. The performance of SMFA, S2MFA, SDMFA, and S2DMFA dimensionality reduction methods for classification using real HSI is evaluated in this paper. Experimental results are compared to more conventional methods like probabilistic principal component analysis, factor analysis, mixtures of factor analyzers and deep mixtures of factor analyzers and it is shown that the proposed methods give better results.

This paper presents two dimensionality reduction methods, mixtures of factor analyzers (MFA) and deep mixtures of factor analyzers (DMFA), for classification of hyperspectral image (HSI). DMFA consists of two layers of MFA and can extract more information from HSI than MFA can. The performance of MFA and DMFA dimensionality reduction methods for classification using real HSI is evaluated in this paper. Experimental results are compared to conventional methods like probabilistic principal component analysis and factor analysis and it is shown that MFA and DMFA give better results.

Recently, the Sentinel-2 (S2) satellite constellation was deployed for mapping and monitoring the Earth environment. Images acquired by the sensors mounted on the S2 platforms have three levels of spatial resolution: 10, 20, and 60 m. In many remote sensing applications, the availability of images at the highest spatial resolution (i.e., 10 m for S2) is often desirable. This can be achieved by generating a synthetic high-resolution image through data fusion. To this end, researchers have proposed techniques exploiting the spectral/spatial correlation inherent in multispectral data to sharpen the lower resolution S2 bands to 10 m. In this paper, we propose a novel method that formulates the sharpening process as a solution to an inverse problem. We develop a cyclic descent algorithm called S2Sharp and an associated tuning parameter selection algorithm based on generalized cross validation and Bayesian optimization. The tuning parameter selection method is evaluated on a simulated data set. The effectiveness of S2Sharp is assessed experimentally by comparisons to state-of-the-art methods using both simulated and real data sets.

Hyperspectral unmixing consists of estimating pure material spectra (endmembers) and their corresponding abundances in hyperspectral images. In this paper, a blind nonlinear hyperspectral unmixing algorithm is presented. The algorithm promotes sparse abundance maps using an lq regularizer and assumes that the spectra are mixed according to an extension to generalized bilinear model, called the Fan model. The algorithm is evaluated using both simulated and real hyperspectral data.

Blind hyperspectral unmixing (HU) is the task of jointly estimating the spectral signatures of materials and abundances in hyperspectral images. Most unmixing algorithms assume the linear mixture model, however, nonlinear models have recently gained interest, as they represent more complicated scenes. This letter proposes two blind nonlinear HU algorithms. The former algorithm assumes that the spectra are mixed according to the generalized bilinear model, while the latter assumes an extension to this model, called the Fan model. Both the algorithms use the ℓqregularizer to promote sparse abundances and solve the minimization problems using cyclic descent. The algorithms are evaluated and compared with other unmixing algorithms using both simulated and real data.

Hyperspectral unmixing is a challenging inverse problem that involves determining the fractional abundances of the representive material (endmembers) in each pixel. In this paper, we develop a neural network autoencoder, that dynamically exploits the sparsity of the abundances and enforces the abundance sum constraint (ASC) for hyperspectral unmixing. Instead of using the conventional mean square error (MSE) objective function, we use the spectral information divergence (SID) measure. Experiments are performed using a real hyperspectral dataset and we compare results obtained using both MSE and SID. It is demonstrated by qualitative inspection that using SID gives significantly better results than using MSE.

In this paper, we present a fast blind multitemporal hyperspectral unmixing algorithm, using an l1penalty to promote sparse abundances. The method is able to account for different acquisition conditions of multitemporal images, by allowing the spectral signatures in the different temporal images to vary. The new algorithm is tested on simulated data and applied on real hyperspectral data.

In this paper, hyperspectral data is modeled as a combination of a sparse component, a low rank component and noise. The low rank component is a product of the endmembers and the abundances in an image, and the sparse component is composed of outliers and structured noise. Outliers and structured noise in this context are, e.g. band specific noise, vertical or horizontal artifacts or saturated pixels. Sparse and low rank matrix decomposition (SLR) is a method that has recently been developed for estimating those components. Here, an algorithm based on ℓ1SLR is developed using sparse blind hyperspectral unmixing and soft thresholding. The number of endmembers and the sparsity parameters are selected using the extended Bayesian information criterion (EBIC). The proposed algorithm is evaluated using a real remote sensing hyperspectral image.

In this letter, we propose a method using a 3-D convolutional neural network to fuse together multispectral and hyperspectral (HS) images to obtain a high resolution HS image. Dimensionality reduction of the HS image is performed prior to fusion in order to significantly reduce the computational time and make the method more robust to noise. Experiments are performed on a data set simulated using a real HS image. The results obtained show that the proposed approach is very promising when compared with conventional methods. This is especially true when the HS image is corrupted by additive noise.

Blind hyperspectral unmixing jointly estimates spectral signatures and abundances in hyperspectral images (HSIs). Hyperspectral unmixing is a powerful tool for analyzing hyperspectral data. However, the usual huge size of HSIs may raise difficulties for classical unmixing algorithms, namely, due to limitations of the hardware used. Therefore, some researchers have considered distributed algorithms. In this paper, we develop a distributed hyperspectral unmixing algorithm that uses the alternating direction method of multipliers and ℓ1sparse regularization. The hyperspectral unmixing problem is split into a number of smaller subproblems that are individually solved, and then the solutions are combined. A key feature of the proposed algorithm is that each subproblem does not need to have access to the whole HSI. The algorithm may also be applied to multitemporal HSIs with due adaptations accounting for variability that often appears in multitemporal images. The effectiveness of the proposed algorithm is evaluated using both simulated data and real HSIs.

Non-negative matrix factorization (NMF) has found use in fields such as remote sensing and computer vision where the signals of interest are usually non-negative. Data dimensions in these applications can be huge and traditional algorithms break down due to unachievable memory demands. One is then compelled to consider distributed algorithms. In this paper, we develop for the first time a distributed version of NMF using the alternating direction method of multipliers (ADMM) algorithm and dyadic cyclic descent. The algorithm is compared to well established variants of NMF using simulated data, and is also evaluated using real remote sensing hyperspectral data.

Blind hyperspectral unmixing is the task of jointly estimating the spectral signatures of material in a hyperspectral images and their abundances at each pixel. The size of hyperspectral images are usually very large, which may raise difficulties for classical optimization algorithms, due to limited memory of the hardware used. One solution to this problem is to consider distributed algorithms. In this paper, we develop a distributed sparse hyperspectral unmixing algorithm using the alternating direction method of multipliers (ADMM) algorithm and ℓ1sparse regularization. Each sub-problem does not need to have access to the whole hyperspectral image. The algorithm is evaluated using a very large real hyperspectral image.

Big data applications, such as medical imaging and genetics, typically generate datasets that consist of few observations n on many more variables p, a scenario that we denote asp ≫ n. Traditional data processing methods are often insufficient for extracting information out of big data. This calls for the development of new algorithms that can deal with the size, complexity, and the special structure of such datasets. In this paper, we consider the problem of classifying p ≫ n data and propose a classification method based on linear discriminant analysis (LDA). Traditional LDA depends on the covariance estimate of the data, but when p ≫ n, the sample covariance estimate is singular. The proposed method estimates the covariance by using a sparse version of noisy principal component analysis (nPCA). The use of sparsity in this setting aims at automatically selecting variables that are relevant for classification. In experiments, the new method is compared to state-of-the art methods for big data problems using both simulated datasets and imaging genetics datasets.

Pansharpening is the process of fusing a high-resolution panchromatic image and a low-spatial-resolution multispectral image to yield a high-spatial-resolution multispectral image. This is a typical ill-posed inverse problem, and in the past two decades, many methods have been proposed to solve it. Still, there is no general consensus on the best way to quantitatively evaluate the spectral and spatial quality of the fused image. In this paper, we compare the two most widely used and accepted methods for quality evaluation. The first method is the verification of the synthesis property which states that the fused image should be as identical as possible to the multispectral image that the sensor would observe at a higher resolution. This is impossible to verify unless the observed images are spatially degraded so that the original observed multispectral image can be used as reference. The second method is to use metrics that do not use a reference, such as the quality no reference (QNR) metrics. However, there is another property, i.e., the consistency property, which states that the fused image reduced to the resolution of the original multispectral image should be as identical to the original image as possible. This has generally been considered a necessary condition that does not have to imply correct fusion. Using real WorldView-2 and QuickBird data and a total of 18 component substitution and multiresolution analysis methods, we demonstrate that the consistency property can indeed be used to give reliable assessment of the relative performance of pansharpening methods and is superior to using the QNR metrics.

In this paper, a novel feature extraction method, called orthogonal total variation component analysis (OTVCA), is proposed for remotely sensed hyperspectral data. The features are extracted by minimizing a total variation (TV) penalized optimization problem. The TV penalty promotes piecewise smoothness of the extracted features which is useful for classification. A cyclic descent algorithm called OTVCA-CD is proposed for solving the minimization problem. In the experiments, OTVCA is applied on a rural hyperspectral image having low spatial resolution and an urban hyperspectral image having high spatial resolution. The features extracted by OTVCA show considerable improvements in terms of classification accuracy compared with features extracted by other state-of-the-art methods.

Blind hyperspectral unmixing involves jointly estimating endmembers and fractional abundances in hyperspectral images. An endmember is the spectral signature of a specific material in an image, while an abundance map specifies the amount of a material seen in each pixel in an image. In this paper, a new cyclic descent algorithm for blind hyperspectral unmixing using total variation (TV) and ℓq sparse regularization is proposed. Abundance maps are both spatially smooth and sparse. Their sparsity derives from the fact that each material in the image is not represented in all pixels. The abundance maps are assumed to be piecewise smooth since adjacent pixels in natural images tend to be composed of similar material. The TV regularizer is used to encourage piecewise smooth images, and the ℓqregularizer promotes sparsity. The dyadic expansion decouples the problem, making a cyclic descent procedure possible, where one abundance map is estimated, followed by the estimation of one endmember. A novel debiasing technique is also employed to reduce the bias of the algorithm. The algorithm is evaluated using both simulated and real hyperspectral images.

In the past two decades, many methods have been proposed to fuse low resolution multispectral (MS) and high resolution panchromatic (Pan) images, i.e., pansharpening. Two large families of such methods are component substitution (CS) and multiresolution analysis methods (MRA). We develop a model based method for pansharpening based on minimizing a cost function which includes a data fidelity term, a detail injection term and a total variation (TV) term. The model takes into account the modulation transfer function (MTF) and spectral response of the sensor. The resulting iterative method not only sharpens the MS image with details from the Pan image but is also able to extract important information from the MS image itself via MTF-based deconvolution. We compare the proposed method to a number of state-of-the-art CS and MRA pansharpening methods using a real WorldView-2 dataset and show that it gives excellent results with details that all the CS and MRA methods can not extract.

The fusion of low resolution multispectral (MS) images and high resolution panchromatic (PAN) images, i.e., pansharpening, is an important technique in remote sensing and has many applications where high resolution imagery is important. Component substitution (CS) and multiresolution analysis (MRA) are two large families of pansharpening methods that are fast and computationally efficient. They can be described using a general framework, where details from the PAN image are added to the upsampled and interpolated MS image. However, these methods often suffer from spectral and spatial distortions. We propose a pre-processing step, where instead of just interpolating the MS image to the resolution scale of the PAN image, we do a deconvolution of the interpolated MS image based on the sensor's modulation transfer function (MTF). This results in large improvement gains in the spectral and spatial quality of the fused image. We demonstrate our method using a real WorldView-2 dataset and show that our approach significantly improves the tested methods in both the CS and MRA families of pansharpening methods.

Blind hyperspectral unmixing jointly estimates both the endmembers and the abundances of hyperspectral images. The endmembers represent the spectral signatures of material found in the image and the abundances specify the amount of each material seen in each pixel in the image. In this paper, a blind hyperspectral unmixing method for feature extraction and classification using total variation (TV) and ℓqsparse regularization is proposed. The abundances found are used as features for classification. The classification results are compared to results obtained using Principal Component analysis (PCA) and also to results obtained using hyperspectral unmixing using only TV and sparsity, respectively.

The identification of the signal subspace is a very important first step for most hyperspectral algorithms. In this letter, we investigate the important problem of identifying the hyperspectral signal subspace by minimizing the mean squared error (MSE) between the true signal and an estimate of the signal. Since it is dependent on the true signal, the MSE is uncomputable in practice, and so we propose a method based on Stein's unbiased risk estimator that provides an unbiased estimate of the MSE. The resulting method is simple and fully automatic, and we evaluate it using both simulated and real hyperspectral data sets. Experimental results show that our proposed method compares well to recent state-of-the-art subspace identification methods.

In remote sensing, due to cost and complexity issues, multispectral (MS) and hyperspectral (HS) sensors have significantly lower spatial resolution than panchromatic (PAN) images. Recently, the problem of fusing coregistered MS and HS images has gained some attention. In this paper, we propose a novel method for fusion of MS/HS and PAN images and of MS and HS images. MS and, more so, HS images contain spectral redundancy, which makes the dimensionality reduction of the data via principal component (PC) analysis very effective. The fusion is performed in the lower dimensional PC subspace; thus, we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and very high tolerance to noise in the observed data. Examples are presented using WorldView 2 data and a simulated data set based on a real HS image, with and without added noise.

There are two main approaches to independent component analysis (ICA); maximization of non-Gaussianity of the sources and the exploitation of temporal correlation in Gaussian sources. In this paper, we present a novel sparse noisy ICA model where we have introduced temporal correlation in the sources, described by a first order auto regressive (AR(1)) process. The correlation structure of the sources eliminates the rotational invariance of the estimates, enabling their separation. Using simulated data, we demonstrate both source separation and denoising, where we compare our results to a sparse PCA method and the fastICA method. Additionally, we apply the method on a real hyperspectral dataset.

Due to cost and complexity issues, hyperspectral (HS) images have lower spatial resolution than multispectral (MS) and panchromatic (PAN) images. We present a novel method for fusing both MS and PAN images and also HS and MS images, based on their statistical properties in the wavelet domain. HS images contain spectral redundancy that makes the dimensionality reduction of the data via principal component analysis (PCA) very effective. The fusion is performed in the lower dimensional PC subspace so we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and a very high tolerance to noise in the observed data. Examples are presented using World View 2 data and also a simulated dataset based on a real HS image, with and without noise.

Hyperspectral images (HSI) are often corrupted by noise making their analysis and interpretation difficult. In this paper we develop a sparse low rank model for HSI, which is useful for denoising. The two key benefits of the model for denoising are dimensionality reduction via noisy principal component analysis (nPCA) and the exploitation of sparse-ness in the dual-tree complex wavelet transform (CWT) coefficients of the loading matrix associated with the principal components (PCs). We present denoising examples of both synthetic and real data and compare our method to a PCA based 2-dimensional (2D) bivariate shrinkage method.

In this paper, a hyperspectral feature extraction method is proposed. A low-rank linear model using the right eigenvector of the observed data is given for hyperspectral images. A total variation (TV) based regularization called Low-Rank TV regularization (LRTV) is used for hyperspectral feature extraction. The feature extraction is used for hyperspectral image classification. The classification accuracies obtained are significantly better than the ones obtained using features extracted by Principal Component Analysis (PCA) and Maximum Noise Fraction (MNF).

Mean squared error (MSE) is commonly used for evaluating the performance of hyperspectral imaging (HSI) methods. MSE depends on the true (unknown) signal to be estimated and is therefore not computable for real data. Therefore, HSI methods are usually evaluated using simulated data. Stein's unbiased risk estimator (SURE) is an unbiased estimator of the MSE that does not require knowledge of the true signal. The main aim of this paper is to promote the use of SURE for evaluating HSI models. To achieve that goal we compare three wavelet models, spectral, spatial and spectral-spatial, for hyperspectral images. Hyperspectral images are modeled based on their sparse wavelet components. The penalized least squares with i.e. penalty (to promote sparsity) is considered for sparse reconstruction. By comparing the SURE values for the three models, it is shown that the spatial model performs better than spectral model and spectral-spatial model outperforms both spectral and spatial models.

In this paper, an effective method is proposed that combines supervised and unsupervised unmixing. We assume a linear model for the hyperspectral data and incorporate information about endmembers that are known to be in the data into the model. This information can be acquired from a spectral library or extracted from the data. Utilizing a priori information can both improve the unmixing, and reduce the complexity of the problem. The method is quantitatively evaluated using simulated data and it is shown that the unmixing results improve and the computational time decreases when a priori information is used. The method is also applied on a real hyperspectral data set of an urban landscape. The estimated abundance maps improve when information about known endmembers is incorporated into the model.

In this letter, we present a new method for the pansharpening of multispectral satellite imagery. Pansharpening is the process of synthesizing a high spatial resolution multispectral image from a low spatial resolution multispectral image and a high-resolution panchromatic (PAN) image. The method uses total variation to regularize an ill-posed problem dictated by a widely used explicit image formation model. This model is based on the assumptions that a linear combination of the bands of the pansharpened image gives the PAN image and that a decimation of the pansharpened image gives the original multispectral image. Experimental results are based on two real datasets and the quantitative quality of the pansharpened images is evaluated using a number of spatial and spectral metrics, some of which have been recently proposed and do not need a reference image. The proposed method compares favorably to other well-known methods for pansharpening and produces images of excellent spatial and spectral quality.

A robust framework for the classification of hyperspectral images which takes into account both spectral and spatial information is proposed. The extended multivariate attribute profile (EMAP) is used for extracting spatial information. Moreover, for solving the so-called curse of dimensionality, supervised feature extraction is carried out on both the original hyperspectral data and the output of the EMAP. After performing the dimensionality reduction, two output vectors of the original data and attributes are concatenated into one stacked vector. The final classification map is achieved by using a random-forest classifier. The main difficulties of using an EMAP is to initialize the attribute parameters. Therefore, a fully automatic scheme of the proposed method is introduced to overcome the shortcomings of using EMAP. The proposed method is tested on two widely known data sets. Experimental results confirm that the proposed method provides an accurate classification map in an acceptable CPU processing time.

In this paper, a method called wavelet-based sparse reduced-rank regression (WSRRR) is proposed for hyperspectral image restoration. The method is based on minimizing a sparse regularization problem subject to an orthogonality constraint. A cyclic descent-type algorithm is derived for solving the minimization problem. For selecting the tuning parameters, we propose a method based on Stein's unbiased risk estimation. It is shown that the hyperspectral image can be restored using a few sparse components. The method is evaluated using signal-to-noise ratio and spectral angle distance for a simulated noisy data set and by classification accuracies for a real data set. Two different classifiers, namely, support vector machines and random forest, are used in this paper. The method is compared to other restoration methods, and it is shown that WSRRR outperforms them for the simulated noisy data set. It is also shown in the experiments on a real data set that WSRRR not only effectively removes noise but also maintains more fine features compared to other methods used. WSRRR also gives higher classification accuracies.

Hyperspectral unmixing is an important technique for analyzing remote sensing images. In this paper, we consider and examine the ℓq, 0 ≤ q ≤ 1 penalty on the abundances for promoting sparse unmixing of hyperspectral data. We also apply a first-order roughness penalty to promote piecewise smooth end-members. A novel iterative algorithm for simultaneously estimating the end-members and the abundances is developed and tested both on simulated and two real hyperspectral data sets. We present an extensive simulation study where we vary both the SNR and the sparsity of the simulated data and identify the model parameters that minimize the reconstruction errors and the spectral angle distance. We show that choosing 0 ≤ q1penalty when the SNR is low or the sparsity of the underlying model is high. We also examine the effects of the imposing the abundance sum constraint using a real hyperspectral data set.

In this paper we consider pansharpening of multispectral satellite imagery based on solving an under-determined inverse problem regularized by the ℓ1-norm of the coefficients of overcomplete multi-scale transforms which all are tight-frame systems. There are two main approaches in sparsity promoting ℓ1-norm regularization, the analysis and the synthesis approach. We perform a number of experiments using two real and well known datasets where the focus is the comparison of the two approaches. One dataset includes a high resolution reference image while the other needs to be degraded prior to pansharpening in order to use the original multispectral image as the reference. Experiments are performed for a range of values for the regularization parameter, where each resulting pansharpened image is evaluated using three quality metrics. The behavior of those metrics as a function of the regularization parameter is compared for the analysis and synthesis formulations and it is shown that analysis gives better results.

This paper deals with hyperspectral image reconstruction using a new linear model and Sparse Regularization (SR). The new model is based on Principal Components (PCs) and wavelets. Since the hyperspectral PCs are not spatially sparse, wavelet is applied to get spatially sparse representation. Sparse regularization is used to recover the corrupted signal. The regularization parameter is chosen by Stein's Unbiased Risk Estimator (SURE). The results show improvements for simulated data sets compare to other denoising methods based on Signal to Noise Ratio (SNR). In addition, the methods are applied on a real noisy data set, and the results of the new method demonstrate visual improvement. The proposed approach is automatic, fast and has the ability to be applied on very large data sets.

We propose CUR methods for hyperspectral unmixing that decompose the data matrix into non-negative endmembers and abundance maps. The endmembers will be selected from a dictionary constructed from the data matrix. Each endmember will coincide with certain columns of the data matrix. By doing this we are assured that the dictionary will be physically meaningful and may be interpreted unambiguously from the data set. This assumption, that the endmembers are contained within the data, is called the pixel purity assumption. We compare two regularization terms to promote sparsity in our solutions, the first is ℓ2regularization and the second is vector ℓ0regularization. The methods are evaluated both on simulated data and a real hyperspectral image of an urban landscape.

Hyperspectral unmixing is the task of decomposing hyperspectral images into endmembers and their abundances. The endmembers are spectral signatures of specific material in the image and the abundances dictate the amount of the material found in each pixel. In this paper we present a blind signal separation method, based on the total variation penalty, that simultaneously estimates the endmembers and the abundances. We evaluate our method using both simulated and a real data set.

Images obtained using Synthetic Aperture Radar (SAR) are corrupted by speckle. Speckle noise results from the chaotic interference of backscattered electromagnetic waves and makes the analysis, interpretation and classification of SAR images difficult. In this paper, we present a denoising algorithm based on Total Variation (TV) regularization. While this kind of denoising algorithm is not new, we propose to select the regularization parameter by minimizing the estimate of the mean square error (MSE) between the denoised image and the clean image. We do not have to know the clean image because we use a statistically unbiased MSE estimate - Stein's Unbiased Risk Estimate (SURE), that depends on the observed image and the estimated image. However, since it is difficult to derive SURE analytically for this kind of problem, we estimate SURE using stochastic methods. We present results using both a simulated image and real SAR image.

In this paper we present a new method for the pansharpening of multi-spectral satellite imagery. This method is based on a simple explicit image formation model which leads to an ill posed problem that needs to be regularized for best results. We use both Tikhonov (ridge regression) and Total Variation (TV) regularization. We develop the solutions to these two problems and then we address the problem of selecting the optimal regularization parameter λ. We find the value of λ that minimizes Stein's unbiased risk estimate (SURE). For ridge regression this leads to an analytical expression for SURE while for the TV regularized solution we use Monte Carlo SURE where the estimate is obtained by stochastic means. Finally, we present experiment results where we use quality metrics to evaluate the spectral and spatial quality of the resulting pansharpened image.

In this paper, we propose a denoising method for hyperspectral images using 3D wavelets. We use the sparse analysis regularization using a 3D overcomplete wavelet dictionary. The minimization problem is solved using iterative Chambolle algorithm. The simulation results show that the 3D dictionary outperforms the 2D one, in terms of Peak Signal to Noise Ratio (PSNR). Denosing hysperspectral cubes is likely to increase the classification accuracy of the hyperspectral data since it can enhance the spectral profiles (or features) that can be useful to discriminate between information classes.

Hyperspectral unmixing is the process where the reflectance spectrum from a mixed pixel is decomposed into separate distinct spectral signatures (endmembers). A mixed pixel results when spectra from more than one material is recorded by a sensor in one pixel. The goal of linear unmixing is to identify the number of endmembers in an image, the endmembers themselves and their abundances in each pixel. This paper presents a new smooth method for unmixing hyperspectral images using nonnegative cyclic descent. The proposed method uses iterative cyclic descent algorithm to find the endmembers and their abundances. The algorithm uses an ℒ1norm to promote sparseness in the abundances. Because the spectrum of the endmembers varies smoothly, a first order roughness penalty is added to discourage roughness in the endmembers. The algorithm does not use any prior information about the data. The method is tested using a real hyperspectral image of an urban landscape.

A new generation of ToF cameras are practical devices capable of real-time 3D scene reconstruction. They are mainly limited by two factors; accuracy and low spatial resolution. Standard CCD and ToF images of a scene share likenesses in some regions, e.g., object edges are apparent as gradients in both modals. Here we propose an approach to overcome the resolution limitation; using an effective dual-tree complex wavelet transform framework in a calibrated setup to fuse the low resolution TOF image with the high resolution details of the CCD image. We show how this can enhance features such as at object borders.

Classification of high resolution urban remote sensing imagery is addressed. The classification is done by both considering the panchromatic imagery and the multi-spectral image obtained using the spectrally consistent fusion method introduced in [1]. The data are classified using support vector machines (SVM). To further enhance the classification accuracy, mathematical morphology is used to derive local spatial information from the panchromatic data. In particular we use the Morphological Profile (MP) in classification of satellite imagery as was proposed in [2, 3]. We also use the derivative of the MP (DMP). In the majority of the image fusion (pansharpening) techniques proposed today, there is a compromise between the spatial enhancement and the spectral consistency. By comparing classification results obtained by using our model based scheme [1] to results obtained using the IHS and Brovey fusion methods, we find that spectrally consistent data give better results when it comes to classification.

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretation, analysis and classification of SAR images harder. Therefore, some speckle reduction is necessary prior to the processing of SAR images. The speckle noise can be modeled as multiplicative i.i.d. Rayleigh noise. Sveinsson and Benediktsson [1996], proposed an adaptive sigmoid thresholding method for SAR images in the wavelet domain. The coefficients thresholding for this method is based on the choice of parameters in the sigmoid thresholding function. They were chosen according to a visual appreciation, i.e., by anad hocmethod. We propose to select these parameters by minimizing an estimate of square error between the clean image and the denoised one. The key point is that we have in our proposal computable, statistically unbiased, MSE estimate - Stein's Unbiased Risk Estimate (SURE) - that depends on the noisy image alone, not on the clean image. We apply the proposed method on an SAR images, both simulated and real data.

The nonsubsampled contourlet transform (NSCT) is a new image representation approach that has sparser representation at both spatial and directional resolution and thus captures smooth contours in images. On the other hand, wavelet transform has sparser representation of homogeneous areas. In this paper, we are going to use the three combinations of undecimated wavelet and nonsubsampled contourlet transforms that was used in for denoising of TerraSAR-X images. Two of the methods use the undecimated wavelet transform to de-noise homogeneous areas and the nonsubsampled contourlet transform to denoise areas with edges. The segmentation between homogeneous areas and areas with edges is done by using total variation segmentation. The third method is a linear averaging of the two denoising methods. A thresholding in the wavelet and contourlet domain is done by non-linear functions which are adapted for each selected subband. The non-linear functions are based on sigmoid functions. Simulation results suggested that these denoising schemes achieve good and clean images.

An approach for spectral-spatial classification of multisource remote sensing data from agricultural areas is addressed. Mathematical morphology is used to derive the spatial information from the data sets. The different data sources (i.e., SAR and multispectral) are classified by support vector machines (SVM). Afterwards, the SVM outputs are transferred to probability measurements. These probability values are combined by different fusion strategies, to derive the final classification result. Comparing the results based on mathematical morphology the total accuracy increased by 6% compared to the pure-pixel classification results. Moreover the transfer of the SVM outputs into probability values and the subsequent fusion further increases the classification accuracy, resulting in an accuracy of 78.5%.

The classification of hyperspectral data is addressed using a classifier ensemble based on Support Vector Machines (SVM). First of all, the hyperspectral data set is decomposed into few sources according to the spectral bands correlation. Then, each source is treated separately and classified by an SVM classifier. Finally, all outputs are used as inputs for the final decision fusion, performed by an additional SVM classifier. The results of experiments, clearly show that the proposed SVM-based decision fusion outperforms a single SVM classifier in terms of overall accuracies.

The nonsubsampled contourlet transform (NSCT) is a new image representation approach that has sparser representation at both spatial and directional resolution and thus captures smooth contours in images On the other hand, wavelet transform has sparser representation of homogeneous areas. In this paper, three combinations of undecimated wavelet and nonsubsampled contourlet transforms will be used for denoising of SAR images. Two of the methods use the wavelet transform to denoise homogeneous areas and the nonsubsampled contourlet transform to denoise areas with edges. The segmentation between homogeneous areas and areas with edges is done by using total variation segmentation. The third method is a linear averaging of the two denoising methods. A thresholding in the wavelet and contourlet domain is done by non-linear functions which are adapted for each selected subband. The non-linear functions are based on sigmoid functions. Simulation results suggested that these denoising schemes achieve good and clean images.

Synthetic Aperture Radar (SAR) images are inherently affected by multiplicative speckle noise, which is due to the coherent nature of the scattering phenomenon. This paper deals with the speckle reduction using the bandlet transform combined with the adaptive sigmoid thresholding. The operation needs to provide multiscale transform. We use the Undecimated Discrete Wavelet Transform (UDWT) and apply the bandlet transform on each resulting scale. Numerical tests applied on Lena image contaminated with multiplicative noise show that our method provides improvement both in terms of image visual fidelity and in terms of Peak Signal-to-Noise Ratio (PSNR). Comparisons are made with the standard wavelet transform and the shift invariant discrete time wavelet transform.

A method is proposed for pixel-level satellite image fusion derived directly from a model of the imaging sensor. By design, the proposed method is spectrally consistent. It is argued that the proposed method needs regularization, as is the case for any method for this problem. A framework for pixel neighborhood regularization is presented. This framework enables the formulation of the regularization in a way that corresponds well with our prior assumptions of the image data. The proposed method is validated and compared with other approaches on several data sets. Lastly, the intensity-hue-saturation method is revisited in order to gain additional insight of what implications the spectral consistency has for an image fusion method.

A method is proposed for the classification of urban hyperspectral data with high spatial resolution. The approach is an extension of previous approaches and uses both the spatial and spectral information for classification. One previous approach is based on using several principal components (PCs) from the hyperspectral data and building several morphological profiles (MPs). These profiles can be used all together in one extended MP. A shortcoming of that approach is that it was primarily designed for classification of urban structures and it does not fully utilize the spectral information in the data. Similarly, the commonly used pixelwise classification of hyperspectral data is solely based on the spectral content and lacks information on the structure of the features in the image. The proposed method overcomes these problems and is based on the fusion of the morphological information and the original hyperspectral data, i.e., the two vectors of attributes are concatenated into one feature vector. After a reduction of the dimensionality, the final classification is achieved by using a support vector machine classifier. The proposed approach is tested in experiments on ROSIS data from urban areas. Significant improvements are achieved in terms of accuracies when compared to results obtained for approaches based on the use of MPs based on PCs only and conventional spectral classification. For instance, with one data set, the overall accuracy is increased from 79% to 83% without any feature reduction and to 87% with feature reduction. The proposed approach also shows excellent results with a limited training set.

Different spatial priors for satellite image fusion are evaluated through experiments on three different data sets. The results are judged visually as well as quantified via different image quality metrics on a down-sampled data-set. It is done within our previously proposed spectrally consistent pan- sharpening framework (SCP). This is a per pixel based fusion framework constructed by considering the imaging physics.

Classification of hyperspectral data with high spatial resolution from urban areas is discussed. An approach has been proposed which is based on using several principal components from the hyperspectral data and build morphological profiles. These profiles can be used all together in one extended morphological profile. A shortcoming of the approach is that it is primarily designed for classification of urban structures and it does not fully utilize the spectral information in the data. Similarly, a pixel-wise classification solely based on the spectral content can be performed, but it lacks information on the structure of the features in the image. An extension is proposed in this paper in order to overcome these dual problems. The proposed method is based on the data fusion of the morphological information and the original hyperspectral data: the two vectors of attributes are concatenated. After a reduction of the dimensionality using Decision Boundary Feature Extraction, the final classification is achieved using a Support Vector Machines classifier. The proposed approach is tested in experiments on ROSIS data from urban areas. Significant improvements are achieved in terms of accuracies when compared to results of approaches based on the use of morphological profiles based on PCs only and conventional spectral classification.

Several widely used methods have been proposed for fusing high resolution panchromatic data and lower resolution multi-channel data. However, many of these methods fail to maintain the spectral consistency of the fused high resolution image, which is of high importance to many of the applications based on satellite data. Additionally, most conventional methods are loosely connected to the image forming physics of the satellite image, giving these methods an ad hoc feel. Vesteinsson et al. [1] proposed a method of fusion of satellite images that is based on the properties of imaging physics in a statistically meaningful way and was called spectral consistent panshapening (SCP). In this paper we improve this framework for satellite image fusion by introducing a better image prior, via data-dependent image smoothing. The dependency is obtained via total variation edge detection method.

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretations, analysis and classifications of SAR images harder. Therefore, some speckle reduction is necessary prior to the processing of SAR images. The speckle noise can be modeled as multiplicative i.i.d. Rayleigh noise. The discrete curvelet transform is a new image representation approach that codes image edges more efficiently than the wavelet transform. On the other hand, wavelet transform codes homogeneous areas better than curvelet transform. In this paper, two combinations of time invariant wavelet and curvelet transforms will be used for denoising of SAR images. Both of the methods use the wavelet transform to denoise homogeneous areas and the curvelet transform to denoise areas with edges. The segmentation between homogeneous areas and areas with edges is done by using total variation segmentation. Simulation results suggested that these denoised schemas can achieve good and clean images.

International audience

Several widely used methods have been proposed for fusing high resolution panchromatic data and lower resolution multi-channel data. However, many of these methods fail to maintain spectral consistency of the fused high resolution image, which is of high importance to many of the applications based on satellite data. Additionally, most conventional methods are loosely connected to the image forming physics of the satellite image, giving these methods an ad hoc feel. Vesteinsson et al. (2005) proposed a method of fusion of satellite images that is based on the properties of imaging physics in a statistically meaningful way. The fusion method was called spectral consistent panshapen- ing (SC) and it was shown that spectral consistency was a direct consequence of imaging physics and hence guaranteed by the SCP. In this paper exploit this framework and investigate two smoothing methods of the fused image obtain by SCP. The first smoothing method is based on Markov random field (MRF) model, while the second method uses wavelet domain hidden Markov models (HMM) for smoothing of the SCP fused image.

Two random forest (RF) approaches are explored; the RF-BHC (binary hierarchical classifier) and the RF-CART (classification and regression tree). Both methods are based on a collection (forest) of tree-like classifier systems where the difference is in the way the trees are grown. The BHC approach depends on class separability measures and the Fisher projection, which maximizes the Fisher discriminant where each tree is a class hierarchy, and the number of leaves is the same as the number of classes. The CART approach is based on CART-like trees where trees are grown to minimize an impurity measure. Here, these different RF approaches are compared in experiments. The RF approaches were investigated in experiments by classification of an urban area from Pavia, Italy using hyperspectral ROSIS (reflective optics system imaging spectrometer) data provided by DLR.

Classification of high-resolution hyperspectral data is investigated. Previously, in classification of high-resolution panchromatic data, simple morphological profiles have been constructed with a repeated use of morphological opening and closing operators with a structuring element of increasing size, starting with the original panchromatic image. This approach has recently been extended for hyperspectral data. In the extension, principal components of the hyperspectral imagery have been computed in order to produce an extended morphological profile. In this paper, we investigate the use of independent components instead of principal components in extended morphological profiles, i.e., selected independent components are used as base images for an extended morphological profile. In the proposed approach, the extended morphological profiles based on the independent components are used as inputs to a neural network classifier. In experiments, a hyperspectral data sets from an urban area in Pavia, Italy is classified.

Abstract — A method for street tracking is proposed. The method consists of two steps. First, a “blob image ” of possible street candidates is created. Then, the street segments from that blob image are extracted. Two feature extraction approaches based on mathematical morphology are applied as preprocessing for the street tracking. One method is based on using differential morphological profiles but the other uses morphological opening and closing operators with a rotating structuring element (SE). The method is tested on an AIRSAR image from Los Angeles with and without noise filtering. The obtained results are measured using two indexes; correctness and completeness. Of the two methods used in the feature extraction, the SE rotation appears to give better results. Noise filtering does not have a major effect in street tracking for the AIRSAR image. I.

Classification of hyperspectral data with high spatial resolution from urban areas is investigated. A method based on mathematical morphology for preprocessing of the hyperspectral data is proposed. In this approach, opening and closing morphological transforms are used in order to isolate bright (opening) and dark (closing) structures in images, where bright/dark means brighter/darker than the surrounding features in the images. A morphological profile is constructed based on the repeated use of openings and closings with a structuring element of increasing size, starting with one original image. In order to apply the morphological approach to hyperspectral data, principal components of the hyperspectral imagery are computed. The most significant principal components are used as base images for an extended morphological profile, i.e., a profile based on more than one original image. In experiments, two hyperspectral urban datasets are classified. The proposed method is used as a preprocessing method for a neural network classifier and compared to more conventional classification methods with different types of statistical computations and feature extraction.

Classification of hyperspectral data with high spatial resolution using both spatial and spectral approaches is discussed. The spatial approach is based on mathematical morphology. In the method, several principal components (PCs) from the hyperspectral data are used. From each of the PCs, a morphological profile is built. These profiles are used together in one extended morphological profile, which is then classified with a neural network. The spectral classification approach is based on maximum likelihood classification and nonparametric weighted feature extraction (NWFE). The results from the spectral and spatial modeling are finally fused together using several different fusion rules. Experimental results are given on a hyperspectral data from an urban area.

The use of random forests for classification of multisource data is investigated in this paper. Random Forest is a classifier that grows many classification trees. Each tree is trained on a bootstrapped sample of the training data, and at each node the algorithm only searches across a random subset of the variables to determine a split. To classify an input vector in random forest, the vector is submitted as an input to each of the trees in the forest, and the classification is then determined by a majority vote. The experiments presented in the paper were done on a multisource remote sensing and geographic data set. The experimental results obtained with random forests were compared to results obtained by bagging and boosting methods.

Classification of hyperspectral remote sensing data with support vector machines (SVMs) is investigated. SVMs have shown to perform well in terms of classification accuracies for hyperspectral data sets. On the other hand, the computational burden of SVMs in hyperdimensional space can be quite intense. Therefore, it is important to explore approaches, which lighten the computational burden without sacrificing the overall classification accuracies. Two different feature extraction methods, decision boundary feature extraction and nonparametric weighted feature extraction are tested. The hyperspectral data are split into several "independent data sources". The data from each data source are transformed using feature extraction, then two approaches are investigated. In the first approach the data from all sources are classified together with a multisource SVM kernel. In the second approach, the data are classified separately using classical SVM RBF kernel. The results from the SVMs are then fused for final classification. Results are compared and discussed.

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretations, analysis and classifications of SAR images harder. Therefore, some speckle reduction is necessary prior to the processing of SAR images. The speckle noise can be modeled as multiplicative i.i.d. Rayleigh noise. Logarithmic transformation of SAR images convert the multiplicative noise models to additive noise. In this paper, two combinations of time invariant wavelet and curvelet transforms will be used for denoising of SAR images. The first one is called the combined filtering algorithm (CFA). This method is based on a constrained optimization problem, both in the wavelet and curvelet domains. The second method is called the adaptive combined method (ACM) which uses the wavelet transform to denoise homogeneous areas and the curvelet transform to denoise areas with edges

The use of Support Vector Machines (SVMs) for classification of multisource data is investigated. SVMs have been shown to have difficulties in classifying multiclass data. To over- come that,the multiclass classification problem considered here was reduced to multiple margin-based binary problems. Several possibilities of binary problems were investigated,including one- against-all,all-pairs and nearly random decompositions of the multiclass problem. To combine the outputs from the binary problems three approaches were tested: a) voting schemes,b) two loss functions,and c) decoding function based on condi- tional probability estimation. An extension of the radial basis function kernel for multisource data is also proposed. The kernel concentrates on local distance between features from each data source. The experimental results show the proposed approach to be appropriate for multisource data classification. I.I NTRODUCTION

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretation, analysis and classification of SAR images harder. In this paper we will consider the use of the curvelet transform (CT), for speckle reduction of SAR images. The CT is a new approach for image representation approach that codes image edges more efficiently then the wavelet transform. Edges are very important in image perception and with fewer coefficients to represent edges, a better denoising scheme can be achieved. We will use three denoising methods: Wavelet-domain hidden Markov tree models, hard thresholding of the curvelet coefficients, and an adaptive combined method (ACM) proposed here, which uses the desired aspects of both aforementioned methods.

Wavelet footprints, proposed by Dragotti (2002), are used for speckle reduction of synthetic aperture radar (SAR) images. Wavelet footprints contain all wavelet coefficients associated with a singular structure of a signal. Consequently, the dependency across scales that is inherent in wavelet transformation is eliminated. In the present paper, coefficients of wavelet footprints are thresholded with hard thresholding. The denoising method shows great promise for speckle removal and hence provides good detection performance for SAR based recognition.

Multisource classification methods based on neural networks, statistical modeling, genetic algorithms, and fuzzy methods are considered. For most of these methods, the individual data sources are at first treated separately and classified by either statistical or neural methods. Then, several decision fusion schemes are applied to combine information from the individual data sources. These schemes include weighted consensus theory where the weights of the individual data sources control the influence of the sources in the combined classification. Using all the data sources individually in consensus-theoretic classification can lead to a redundancy in the classification process. Therefore, a special focus in this letter is on neural networks based on pruning and regularization for combination and classification. The considered methods are applied in classification of a multisource dataset.

Two wavelet transformations are used for speckle reduction and enhancement of synthetic aperture radar (SAR) images. First, a discrete wavelet transformation (DWT) based on oversampled filter banks is used. The oversampled DWT is called a double-density DWT (DD-DWT) and is based on a single-scaling function (low pass) and two distinct wavelet functions (high pass). Second, a discrete wavelet transformation based on two dual real wavelet trees is applied. Each tree produces a set of real DWTs, which together form the complex wavelet transformation (CWT), i.e., a transformation with both real and imaginary parts. Both of these DWTs are almost translation invariant and are useful for speckle reduction through their subband images, and the speckle reduction is obtained by thresholding the subband image coefficients of the digitized SAR images. A thresholding method based on the use of nonlinear functions, which are adapted for each selected subband, is used. The nonlinear functions are based on sigmoid functions. The denoising method presented shows great promise for speckle removal and, hence, can provide good detection performance for SAR-based recognition.

Multiple Self-Organizing Feature Maps (MSOMs) can be considered attractive for classification of remote sensing data with many input features. The MSOMs have several advantages, e.g., they are non-parametric, the computational cost for them only grows linearly as a function of the number of features, and they have been shown to approximate posterior probabilities. In the paper MSOMs are investigated for classification of a multisource remote sensing and geographic data set. In the experiments, the MSOM showed potential for classification of the multisource data set.

Discrete wavelet transformations (DWTs) based on oversampled filter banks, proposed by I. W. Selesnick (2001), are used for speckle reduction of SAR images. The oversampled DWT is called double density DWT (DD-DWT) and is based on a single scaling function (lowpass) and two distinct wavelet functions (highpass). The DD-DWT is useful for speckle reduction through its subband images and the speckle reduction is obtained by thresholding the subband-image coefficients of the digitized SAR images. A thresholding method using non-linear functions which are adapted for each selected subband is used in the paper. The non-linear functions are based on sigmoid functions. The denoising method shows great promise for speckle removal and hence can provide good detection performance for SAR based recognition.

A linear feature extraction method based on the discrete wavelet transform (DWT) is applied. A binary genetic algorithm is used to select the best features from the different DWT representations in terms of cost. The feature extraction/selection methods are applied in classification of multisource remote sensing and geographic data. In experiments, the proposed methods performed well in terms of overall accuracies as compared to results obtained with other well-known feature extraction/selection methods.

Curvelet transform (CT), proposed by E. Candes et al. (1999), is used for speckle reduction of SAR images. The CT is useful for speckle reduction through its subband images and the speckle reduction is obtained by thresholding the subband-image coefficients of the digitized SAR images. Two thresholding methods are used; hard thresholding and soft thresholding. The denoising method shows great promise for speckle removal and hence provides good detection performance for SAR based recognition.

The combination of multisource remote sensing and geographic data is believed to offer improved accuracies in land cover classification. For such classification, the conventional parametric statistical classifiers, which have been applied successfully in remote sensing for the last two decades, are not appropriate, since a convenient multivariate statistical model does not exist for the data. In this paper, several single and multiple classifiers, that are appropriate for the classification of multisource remote sensing and geographic data are considered. The focus is on multiple classifiers: bagging algorithms, boosting algorithms, and consensus-theoretic classifiers. These multiple classifiers have different characteristics. The performance of the algorithms in terms of accuracies is compared for two multisource remote sensing and geographic datasets. In the experiments, the multiple classifiers outperform the single classifiers in terms of overall accuracies.

Hybrid classification methods based on consensus from several data sources are considered. Each data source is at first treated separately and modeled using statistical methods. Then weighting mechanisms are used to control the influence of each data source in the combined classification. The weights are optimized in order to improve the combined classification accuracies. Both linear and nonlinear optimization methods are considered and used in classification of two multisource remote sensing and geographic data sets. A nonlinear method which utilizes a neural network gives excellent experimental results. The hybrid statistical/neural method outperforms all other methods in terms of test accuracies in the experiments.

A new type of a neural-network architecture, the parallel consensual neural network (PCNN), is introduced and applied in classification/data fusion of multisource remote sensing and geographic data. The PCNN architecture is based on statistical consensus theory and involves using stage neural networks with transformed input data. The input data are transformed several times and the different transformed data are used as if they were independent inputs. The independent inputs are first classified using the stage neural networks. The output responses from the stage networks are then weighted and combined to make a consensual decision. In this paper, optimization methods are used in order to weight the outputs from the stage networks. Two approaches are proposed to compute the data transforms for the PCNN, one for binary data and another for analog data. The analog approach uses wavelet packets. The experimental results obtained with the proposed approach show that the PCNN outperforms both a conjugate-gradient backpropagation neural network and conventional statistical methods in terms of overall classification accuracy of test data.

Statistical classification methods based on consensus from several data sources are considered. The methods need weighting mechanisms to control the influence of each data source in the combined classification. The weights are optimized in order to improve the combined classification accuracies. Both linear and non-linear methods are considered for the optimization. A non-linear method which utilizes a neural network is proposed and gives excellent results in experiments. Consensus theory optimized with neural networks outperforms all other methods both in terms of training and test accuracies in the experiments.

The processing of Airborne Visible-Infrared Imaging Spectrometer (AVIRIS) data is discussed both in terms of feature extraction and classification. The recently proposed decision boundary feature extraction method is reviewed and then applied in experiments. Results of classifications for AVIRIS data from Iceland 1991 are given with emphasis on geological applications. The classifiers used include neural network methods and statistical approaches. The decision boundary feature extraction method shows excellent performance for these data.>