Observation of the Earth using satellites mounted with optical sensors is an important application of remote sensing. Owing to physical constraints, multispectral (MS) sensors acquire images of lower spatial resolution than a single-band panchromatic (PAN) sensor that acquires images of the same scene. Pansharpening fuses the MS and PAN images to obtain an MS image with the same spatial resolution as the PAN image. In this letter, we propose to expand a method, initially developed for Sentinel-2 single-sensor sharpening, for pansharpening. The expanded method is based on solving a non-convex MS acquisition model using optimization methods based on cyclic decent and manifold optimization. The tuning parameters of the method are chosen using Bayesian optimization with reduced-scale evaluation. The proposed method is compared with a number of established pansharpening methods and is validated using both synthetic and real data sets.

The fusion of a low spatial resolution multispectral image and a high spatial resolution panchromatic image, i.e., pan-sharpening is an important technique in remote sensing where high resolution imagery is needed. Two of the largest families of such methods are the component substitution (CS) and multi-resolution analysis (MRA) methods. These families of methods can be described by general detail injection schemes which are closely related. In this paper, we propose pansharpening methods which are based on directly implementing these schemes using a convolutional neural network (CNN) such that the mean squared error between the down-sampled fused image and the observed multispectral image is minimized. Using a simulated Pleiades dataset we demonstrate that the proposed approach gives excellent results when compared to other state-of-the-art CS, MRA and CNN methods.

Recently, the Sentinel-2 (S2) satellite constellation was deployed for mapping and monitoring the Earth environment. Images acquired by the sensors mounted on the S2 platforms have three levels of spatial resolution: 10, 20, and 60 m. In many remote sensing applications, the availability of images at the highest spatial resolution (i.e., 10 m for S2) is often desirable. This can be achieved by generating a synthetic high-resolution image through data fusion. To this end, researchers have proposed techniques exploiting the spectral/spatial correlation inherent in multispectral data to sharpen the lower resolution S2 bands to 10 m. In this paper, we propose a novel method that formulates the sharpening process as a solution to an inverse problem. We develop a cyclic descent algorithm called S2Sharp and an associated tuning parameter selection algorithm based on generalized cross validation and Bayesian optimization. The tuning parameter selection method is evaluated on a simulated data set. The effectiveness of S2Sharp is assessed experimentally by comparisons to state-of-the-art methods using both simulated and real data sets.

Hyperspectral unmixing is a challenging inverse problem that involves determining the fractional abundances of the representive material (endmembers) in each pixel. In this paper, we develop a neural network autoencoder, that dynamically exploits the sparsity of the abundances and enforces the abundance sum constraint (ASC) for hyperspectral unmixing. Instead of using the conventional mean square error (MSE) objective function, we use the spectral information divergence (SID) measure. Experiments are performed using a real hyperspectral dataset and we compare results obtained using both MSE and SID. It is demonstrated by qualitative inspection that using SID gives significantly better results than using MSE.

In this letter, we propose a method using a 3-D convolutional neural network to fuse together multispectral and hyperspectral (HS) images to obtain a high resolution HS image. Dimensionality reduction of the HS image is performed prior to fusion in order to significantly reduce the computational time and make the method more robust to noise. Experiments are performed on a data set simulated using a real HS image. The results obtained show that the proposed approach is very promising when compared with conventional methods. This is especially true when the HS image is corrupted by additive noise.

Big data applications, such as medical imaging and genetics, typically generate datasets that consist of few observations n on many more variables p, a scenario that we denote asp ≫ n. Traditional data processing methods are often insufficient for extracting information out of big data. This calls for the development of new algorithms that can deal with the size, complexity, and the special structure of such datasets. In this paper, we consider the problem of classifying p ≫ n data and propose a classification method based on linear discriminant analysis (LDA). Traditional LDA depends on the covariance estimate of the data, but when p ≫ n, the sample covariance estimate is singular. The proposed method estimates the covariance by using a sparse version of noisy principal component analysis (nPCA). The use of sparsity in this setting aims at automatically selecting variables that are relevant for classification. In experiments, the new method is compared to state-of-the art methods for big data problems using both simulated datasets and imaging genetics datasets.

Pansharpening is the process of fusing a high-resolution panchromatic image and a low-spatial-resolution multispectral image to yield a high-spatial-resolution multispectral image. This is a typical ill-posed inverse problem, and in the past two decades, many methods have been proposed to solve it. Still, there is no general consensus on the best way to quantitatively evaluate the spectral and spatial quality of the fused image. In this paper, we compare the two most widely used and accepted methods for quality evaluation. The first method is the verification of the synthesis property which states that the fused image should be as identical as possible to the multispectral image that the sensor would observe at a higher resolution. This is impossible to verify unless the observed images are spatially degraded so that the original observed multispectral image can be used as reference. The second method is to use metrics that do not use a reference, such as the quality no reference (QNR) metrics. However, there is another property, i.e., the consistency property, which states that the fused image reduced to the resolution of the original multispectral image should be as identical to the original image as possible. This has generally been considered a necessary condition that does not have to imply correct fusion. Using real WorldView-2 and QuickBird data and a total of 18 component substitution and multiresolution analysis methods, we demonstrate that the consistency property can indeed be used to give reliable assessment of the relative performance of pansharpening methods and is superior to using the QNR metrics.

In the past two decades, many methods have been proposed to fuse low resolution multispectral (MS) and high resolution panchromatic (Pan) images, i.e., pansharpening. Two large families of such methods are component substitution (CS) and multiresolution analysis methods (MRA). We develop a model based method for pansharpening based on minimizing a cost function which includes a data fidelity term, a detail injection term and a total variation (TV) term. The model takes into account the modulation transfer function (MTF) and spectral response of the sensor. The resulting iterative method not only sharpens the MS image with details from the Pan image but is also able to extract important information from the MS image itself via MTF-based deconvolution. We compare the proposed method to a number of state-of-the-art CS and MRA pansharpening methods using a real WorldView-2 dataset and show that it gives excellent results with details that all the CS and MRA methods can not extract.

The fusion of low resolution multispectral (MS) images and high resolution panchromatic (PAN) images, i.e., pansharpening, is an important technique in remote sensing and has many applications where high resolution imagery is important. Component substitution (CS) and multiresolution analysis (MRA) are two large families of pansharpening methods that are fast and computationally efficient. They can be described using a general framework, where details from the PAN image are added to the upsampled and interpolated MS image. However, these methods often suffer from spectral and spatial distortions. We propose a pre-processing step, where instead of just interpolating the MS image to the resolution scale of the PAN image, we do a deconvolution of the interpolated MS image based on the sensor's modulation transfer function (MTF). This results in large improvement gains in the spectral and spatial quality of the fused image. We demonstrate our method using a real WorldView-2 dataset and show that our approach significantly improves the tested methods in both the CS and MRA families of pansharpening methods.

In remote sensing, due to cost and complexity issues, multispectral (MS) and hyperspectral (HS) sensors have significantly lower spatial resolution than panchromatic (PAN) images. Recently, the problem of fusing coregistered MS and HS images has gained some attention. In this paper, we propose a novel method for fusion of MS/HS and PAN images and of MS and HS images. MS and, more so, HS images contain spectral redundancy, which makes the dimensionality reduction of the data via principal component (PC) analysis very effective. The fusion is performed in the lower dimensional PC subspace; thus, we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and very high tolerance to noise in the observed data. Examples are presented using WorldView 2 data and a simulated data set based on a real HS image, with and without added noise.

There are two main approaches to independent component analysis (ICA); maximization of non-Gaussianity of the sources and the exploitation of temporal correlation in Gaussian sources. In this paper, we present a novel sparse noisy ICA model where we have introduced temporal correlation in the sources, described by a first order auto regressive (AR(1)) process. The correlation structure of the sources eliminates the rotational invariance of the estimates, enabling their separation. Using simulated data, we demonstrate both source separation and denoising, where we compare our results to a sparse PCA method and the fastICA method. Additionally, we apply the method on a real hyperspectral dataset.

Due to cost and complexity issues, hyperspectral (HS) images have lower spatial resolution than multispectral (MS) and panchromatic (PAN) images. We present a novel method for fusing both MS and PAN images and also HS and MS images, based on their statistical properties in the wavelet domain. HS images contain spectral redundancy that makes the dimensionality reduction of the data via principal component analysis (PCA) very effective. The fusion is performed in the lower dimensional PC subspace so we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and a very high tolerance to noise in the observed data. Examples are presented using World View 2 data and also a simulated dataset based on a real HS image, with and without noise.

Hyperspectral images (HSI) are often corrupted by noise making their analysis and interpretation difficult. In this paper we develop a sparse low rank model for HSI, which is useful for denoising. The two key benefits of the model for denoising are dimensionality reduction via noisy principal component analysis (nPCA) and the exploitation of sparse-ness in the dual-tree complex wavelet transform (CWT) coefficients of the loading matrix associated with the principal components (PCs). We present denoising examples of both synthetic and real data and compare our method to a PCA based 2-dimensional (2D) bivariate shrinkage method.

In this letter, we present a new method for the pansharpening of multispectral satellite imagery. Pansharpening is the process of synthesizing a high spatial resolution multispectral image from a low spatial resolution multispectral image and a high-resolution panchromatic (PAN) image. The method uses total variation to regularize an ill-posed problem dictated by a widely used explicit image formation model. This model is based on the assumptions that a linear combination of the bands of the pansharpened image gives the PAN image and that a decimation of the pansharpened image gives the original multispectral image. Experimental results are based on two real datasets and the quantitative quality of the pansharpened images is evaluated using a number of spatial and spectral metrics, some of which have been recently proposed and do not need a reference image. The proposed method compares favorably to other well-known methods for pansharpening and produces images of excellent spatial and spectral quality.

In this paper we consider pansharpening of multispectral satellite imagery based on solving an under-determined inverse problem regularized by the ℓ1-norm of the coefficients of overcomplete multi-scale transforms which all are tight-frame systems. There are two main approaches in sparsity promoting ℓ1-norm regularization, the analysis and the synthesis approach. We perform a number of experiments using two real and well known datasets where the focus is the comparison of the two approaches. One dataset includes a high resolution reference image while the other needs to be degraded prior to pansharpening in order to use the original multispectral image as the reference. Experiments are performed for a range of values for the regularization parameter, where each resulting pansharpened image is evaluated using three quality metrics. The behavior of those metrics as a function of the regularization parameter is compared for the analysis and synthesis formulations and it is shown that analysis gives better results.

Images obtained using Synthetic Aperture Radar (SAR) are corrupted by speckle. Speckle noise results from the chaotic interference of backscattered electromagnetic waves and makes the analysis, interpretation and classification of SAR images difficult. In this paper, we present a denoising algorithm based on Total Variation (TV) regularization. While this kind of denoising algorithm is not new, we propose to select the regularization parameter by minimizing the estimate of the mean square error (MSE) between the denoised image and the clean image. We do not have to know the clean image because we use a statistically unbiased MSE estimate - Stein's Unbiased Risk Estimate (SURE), that depends on the observed image and the estimated image. However, since it is difficult to derive SURE analytically for this kind of problem, we estimate SURE using stochastic methods. We present results using both a simulated image and real SAR image.

In this paper we present a new method for the pansharpening of multi-spectral satellite imagery. This method is based on a simple explicit image formation model which leads to an ill posed problem that needs to be regularized for best results. We use both Tikhonov (ridge regression) and Total Variation (TV) regularization. We develop the solutions to these two problems and then we address the problem of selecting the optimal regularization parameter λ. We find the value of λ that minimizes Stein's unbiased risk estimate (SURE). For ridge regression this leads to an analytical expression for SURE while for the TV regularized solution we use Monte Carlo SURE where the estimate is obtained by stochastic means. Finally, we present experiment results where we use quality metrics to evaluate the spectral and spatial quality of the resulting pansharpened image.

Classification of high resolution urban remote sensing imagery is addressed. The classification is done by both considering the panchromatic imagery and the multi-spectral image obtained using the spectrally consistent fusion method introduced in [1]. The data are classified using support vector machines (SVM). To further enhance the classification accuracy, mathematical morphology is used to derive local spatial information from the panchromatic data. In particular we use the Morphological Profile (MP) in classification of satellite imagery as was proposed in [2, 3]. We also use the derivative of the MP (DMP). In the majority of the image fusion (pansharpening) techniques proposed today, there is a compromise between the spatial enhancement and the spectral consistency. By comparing classification results obtained by using our model based scheme [1] to results obtained using the IHS and Brovey fusion methods, we find that spectrally consistent data give better results when it comes to classification.