Autonomous Underwater Vehicle (AUV) swarms face a fundamental synchronization challenge where millisecond-scale acoustic propagation delays prevent the nanosecond-level precision required for cooperative MIMO. We present a three-tier Multi-Agent Reinforcement Learning (MARL) framework using Magnetic Induction (MI) for ultra-precise synchronization within 100 m clusters while acoustic channels handle long-range data transmission. The key innovation lies in synergistic dual-modality operation: MI continuously provides nanosecond timing references as a deterministic control plane (8.5 ± 2.1 ns precision, 118× better than the 1 μs threshold), serving as the critical enabler that–combined with accurate channel estimation and positioning–enables acoustic C-MIMO beamforming gains previously unattainable underwater. Our hierarchical architecture employs the novel Delay-Sensitive Asynchronous Value Decomposition (DS-AVD) algorithm, explicitly handling 500 ms acoustic delays through temporal credit assignment and depth-dependent value decomposition. Comprehensive physics-informed simulations incorporating Bellhop acoustic models and hardware-validated MI measurements demonstrate: +12.3 dB beamforming gain, achieving 67% average array gain efficiency across 2-16 AUV configurations, 38.0±1.2 dB median Signal to Noise Ratio (SINR) (31% improvement), 140% range extension (2.1 km to 5.0 km), and 61% improvement of energy efficiency. DS-AVD achieves 95 ± 3% task success rate under nominal conditions (50 − 500 m depth) with graceful degradation to 82 ± 6% at 1000 m depth, where standard MARL approaches achieve only 11 ± 4%. Real-time feasibility is validated on embedded platforms with 47 ms inference latency and 2.3 W power consumption. This simulation-validated framework provides the foundation for next-generation underwater IoT systems that bridge the underwater-terrestrial performance gap.

Core image processing tasks, such as super-resolution, denoising, deblurring, pansharpening, and atmospheric correction, underpin all optical remote sensing (RS) pipelines. Errors at this stage propagate through downstream applications, distorting land-cover maps, change detection, and climate records. Classical physics-based models capture sensor optics, radiometry, and geometry but struggle with complex noise and scene variability. In contrast, deep learning (DL) methods offer powerful data-driven solutions yet often act as closed boxes, ignoring physical constraints and overfitting to spurious patterns. Hybrid DL (HDL) approaches bridge this gap by integrating physical models with neural architectures, combining interpretability and data adaptivity. This article surveys the emerging landscape of HDL methods in RS image processing, outlining their theoretical foundations, motivations, and design philosophies. We categorize fusion strategies, from model-embedded schemes (e.g., plug-and-play (PnP) and unrolling) to model-guided learning (e.g., deep image prior (DIP) and unsupervised frameworks), and discuss how they enhance trust, robustness, and physical consistency in RS image analysis.

This study examines the microwave emission properties of snow-covered sea ice, modeled as a layer with rough top and bottom boundaries. The emission model is based on the first-order solution to the radiative transfer equation (RTE). This equation describes the brightness temperatures that propagate both upward and downward in the layer, and it is solved numerically using the eigenvalue method. Additionally, the model takes into account contributions from irregular boundaries as well as surface and volume scattering interactions. The study looks at the variability of brightness temperature and the growth of different ice types. It uses modulation theory to consider the roughness of the boundaries, showing how this affects the emission from various types of snow-covered sea ice. We used polarization ratio-gradient ratio (PR-GR) space to analyze ice-grown transitions quantitatively. The thresholds of PR were found to be sensitive to changes in ice concentration. It was observed that the threshold of PR did not change with the roughness of the top boundary. We also found that the roughness of the ice surface has a more significant impact on emission than that of the snow-covered ice. It also observed that roughness effects are more potent in H-polarization than in V-polarization for each of the seven ice types. The roughness effects are weaker at large look angles, indicating stronger volume scattering. The study concludes that the presence of roughness in the ice layer’s boundaries leads to noticeable variations in brightness temperature.

A comprehensive exploration of the transformer and dual-receiver system-based direction-of-arrival (DOA) estimation is presented in the context of sea surface scattering, particularly under varying sea conditions. A bidirectional encoder representation from transformer (BERT) with a physics-based loss function is utilized to process two individual channel radars. The datasets are the radar scattering coefficients of sea surface simulated at C-band for copolarizations and cross-polarizations. Through detailed analysis of simulated datasets and root mean square error (RMSE) evaluations, the model’s performance is investigated across different observation modes, namely, the co-polar (CP), co-azimuth (CA), full-bistatic (FB), and Beaufort wind scale from 3 to 5. Our study demonstrates that the bidirectional encoder representation from transformer model, employing a physics-based loss function, outperforms the baseline long short-term memory (LSTM) model, especially under high noise levels and with larger datasets. Significant correlations between wind conditions and DOA accuracy are observed, highlighting the bidirectional encoder representation from transformer model’s adaptability to dynamic environmental factors, particularly under increased wind scales. The choice of observation mode, with CP and FB consistently outperforming CA, proves pivotal. Precise simulation of speckle variations and optimized observation mode selection are identified as crucial avenues for enhancing the model’s practical utility.

The Sentinel-2 (S2) satellite constellation provides images at three different spatial resolutions and model based superresolution methods have proved useful for sharpening them to their maximum resolution. Algorithm unrolling is a way of building efficient, interpretable neural networks by reimplementing traditional algorithms in a neural network context. In this paper, an unrolled model based method to superresolve S2 images is proposed and unsupervised single image training is performed using reduced scale data. The method is evaluated using both real and simulated data.

Microsatellites and drones are often equipped with digital cameras whose sensing system is based on color filter arrays (CFAs), which define a pattern of color filter overlaid over the focal plane. Recent commercial cameras have started implementing RGBW patterns, which include some filters with a wideband spectral response together with the more classical RGB ones. This allows for additional light energy to be captured by the relevant pixels and increases the overall SNR of the acquisition. Demosaicking defines reconstructing a multi-spectral image from the raw image and recovering the full color components for all pixels. However, this operation is often tailored for the most widespread patterns, such as the Bayer pattern. Consequently, less common patterns that are still employed in commercial cameras are often neglected. In this work, we present a generalized framework to represent the image formation model of such cameras. This model is then exploited by our proposed demosaicking algorithm to reconstruct the datacube of interest with a Bayesian approach, using a total variation regularizer as prior. Some preliminary experimental results are also presented, which apply to the reconstruction of acquisitions of various RGBW cameras.

This paper proposes a new loss function to train a convolutional neural network (CNN) for multispectral and hyper-spectral (MS-HS) image fusion. The loss function is based on the relative dimensionless global error synthesis (ER-GAS), where we exchange the mean squared error (MSE) for its unbiased estimate using Stein’s risk unbiased estimate (SURE). The loss function has a good balance between the spectral and spatial information implied by the weighted MSE, therefore it does not need a parameter to balance the spectral and spatial terms as in MSE loss function, and it also converges faster than the MSE one. Additionally, the loss function enables unsupervised training and avoids overfit-ting, since it is derived by using SURE. Experimental results show that the proposed method yields good results and outperforms the competitive methods. Codes are available at https://github.com/hvn2/SURE-ERGAS

Hyperspectral image (HSI) denoising methods that are implemented using deep learning frameworks rarely consider the intrinsic characteristics of HSIs, and often lack both physical interpretability, and generalization. In this paper, a low-rank and sparse model-based unrolled network for HSI de-noising, termed LRS-Net, is proposed. The method unrolls a model-based denoising method into a deep-unrolled network. The network is much faster than the previous method and is also able to automatically select the tuning parameters. The method inherits the advantages of model-based methods, i.e., physical interpretability and generalization, and also advantages from deep learning based methods, i.e., computational efficiency and data-based learning capabilities. Using both simulated and real HSIs it is shown the proposed method can outperform other comparative methods, both in quantitative and visual assessments.

This paper proposes a new spectral-spatial kernel minimum noise fraction transformation (SS-KMNF) as a dimensionality reduction method for hyperspectral image (HSI) classification. The kernel minimum noise fraction (KMNF) method generates new components ordered by image quality and the key index of image quality is noise fraction. In SS-KMNF, the high correlation between bands in homogeneous regions obtained using a superpixel technique is applied to improve the precision of noise fraction. Compared with the original KMNF, SS-KMNF can fully use spectral and spatial information contained in a HSI and is more effective in enhancing the performance of dimensionality reduction for HSI classification. Moreover, a new classification strategy is proposed based on superpixels which are considered as the basic unit instead of pixels for classifying features extracted by SS-KMNF. Experimental results show that SS-KMNF can get better dimensionality reduction performance than KMNF, and the superpixel is not only beneficial to precisely estimating noise fraction but can also increase the classification accuracy of features extracted by SS-KMNF. In addition, SS-KMNF can also be applied to classify transmission lines in electrical power systems.

Multispectral remote sensing images are often have band-dependent image resolution due to cost and technical limitations. To address this, we developed a method that sharpens low-resolution (LR) images using high-resolution (HR) images. In this paper, we propose a novel unsupervised deep learning (DL) approach that involves unrolling an iterative algorithm into a deep neural network and training it using a loss function based on Stein’s risk unbiased estimate (SURE) to sharpen the LR bands (20 and 60 m) of Sentinel-2 (S2) to their highest resolution (10 m). This approach views traditional optimization model-based methods through a DL framework, improving interpretability and clarifying connections between the two approaches. Results from both simulated and real S2 datasets demonstrate that the proposed method outperforms competitive methods and produces high-quality images for the 20 m and 60 m bands. The codes are available at: https://github.com/hvn2/S2-Unrolling.

Recent advances in deep learning (DL) reveal that the structure of a convolutional neural network (CNN) is a good image prior (called deep image prior (DIP)), bridging the model-based and DL-based methods in image restoration. However, optimizing a DIP-based CNN is prone to over-fitting leading to a poorly reconstructed image. This paper derives a loss function based on Stein's unbiased risk estimate (SURE) for unsupervised training of a DIP-based CNN applied to the hyperspectral image (HSI) super-resolution. The SURE loss function is an unbiased estimate of the mean-square-error (MSE) between the clean low-resolution image and the low-resolution estimated image, which relies only on the observed low-resolution image. Experimental results on HSI show that the proposed method not only improves the performance, but also avoids overfitting. Codes are available at https://github.com/hvn2/SURE-MS-HS

Spectral and spatial correlation in hyperspectral images (HSIs) can be exploited in HSI processing because it directly induces a sparse and low-rank prior via linear transformations. Researchers have used the sparse and low-rank prior as an image prior for HSI restoration, such as denoising, deblurring, and super-resolution. This paper proposes a HSI denoising method that incorporates a sparse and low-rank prior with a deep image prior (DIP). The sparse and low-rank prior is obtained using the 2-dimensional discrete wavelet transform (2-D DWT), and singular value decomposition (SVD), while the DIP is provided by the structure of a convolutional neural network (CNN). The combination of a sparse and low-rank prior with a DIP views the CNN-based denoising method similar to a model-based method, inheriting the advantages of both model-based and CNN-based methods. Experimental results with simulated and real HSI datasets show that the proposed method outperforms the conventional sparse and low-rank based methods in both quantitative and qualitative performance. Codes are available at https://github.com/hvn2/DIP-SLR

Spectral variability in hyperspectral images (HSIs) has received lot of attention over the last years, especially in the field of hyperspectral unmixing (HU) where it is a major issue. In this letter, we propose a method utilizing a variational autoencoder (VAE) for creating synthetic HSIs having controllable degree of spectral variability from existing HSIs with established ground-truth abundance maps and endmembers. Such synthetic datasets can be useful for developing HU methods that can handle spectral variability in HSIs. We investigate how the variability in the synthetic images differs from the original images and perform blind unmixing experiments using generated datasets to illustrate the effect of increasing variability. Code for method is available athttps://github.com/burknipalsson/vae_synthetic_hsi.

This letter proposes a semi-supervised mixtures of factor analyzers (S2MFA) feature extraction (FE) method for hyperspectral image (HSI). S2MFA uses a Gaussian mixture model to segment the image to different regions, each region follows a Gaussian distribution and contains labeled and unlabeled samples. The method uses a factor analyzer to get a factor-loading matrix to preserve the local spatial information using the labeled and unlabeled samples. It simultaneously improves the class discrimination of the data using the labeled samples and also transforms the original image to an optimal low-dimensional subspace to achieve dimensionality reduction. The performance of the S2MFA FE method is evaluated by classification of two real HSIs and compared to different kinds of statistic unsupervised, supervised, and semi-supervised FE methods.

Image fusion is utilized in remote sensing due to the limitation of the imaging sensor and the high cost of simultaneously acquiring high spatial and spectral resolution images. Optical remote sensing imaging systems usually provide images of high spatial resolution but low spectral resolution and vice versa. Therefore, fusing those images to obtain a fused image having both high spectral and spatial resolution is desirable in many applications. This paper proposes a fusion framework using an unsupervised convolutional neural network (CNN) and Stein’s unbiased risk estimate (SURE). We derive a new loss function for a CNN that incorporates back-projection mean-squared error with SURE to estimate the projected mean-square-error (MSE) between the fused image and the ground truth. The main motivation is that training a CNN with this SURE loss function is unsupervised and avoids overfitting. Experimental results for two fusion examples, multispectral and hyperspectral (MS-HS) image fusion, and multispectral and multispectral (MS-MS) image fusion, show that the proposed method yields high quality fused images and outperforms the competitive methods. Codes are be available at https://github.com/hvn2/Deep-SURE-Fusion.

This article proposes a denoising method based on sparse spectral–spatial and low-rank representations (SSSLRR) using the 3-D orthogonal transform (3-DOT). SSSLRR can be effectively used to remove the Gaussian and mixed noise. SSSLRR uses 3-DOT to decompose noisy HSI to sparse transform coefficients. The 3-D discrete orthogonal wavelet transform (3-D DWT) is a representative 3-DOT suitable for denoising since it concentrates on the signal in few transform coefficients, and the 3-D discrete orthogonal cosine transform (3-D DCT) is another example. An SSSLRR using 3-D DWT will be called SSSLRR-DWT. SSSLRR-DWT is an iterative algorithm based on the alternating direction method of multipliers (ADMM) that uses sparse and nuclear norm penalties. We use an ablation study to show the effectiveness of the penalties we employ in the method. Both simulated and real hyperspectral datasets demonstrate that SSSLRR outperforms other comparative methods in quantitative and visual assessments to remove the Gaussian and mixed noise.

In recent years numerous model-based methods for super-resolution of Sentinel-2 (S2) multispectral images have been suggested. Super-resolution aims to enhance the resolution of a captured image by upscaling and enhancing the details. The performance of model-based methods relies on carefully selecting regularizers and tuning parameters. This paper investigates whether using Wald's protocol, i.e., selecting tuning parameters at reduced-resolution, translates to a good performance at a full-scale. To investigate this, we use the recently proposed S2Sharp method and show that selecting its tuning parameters using Wald's protocol improves its performance.

This paper proposes a novel method for sharpening the 20 m bands of the multispectral images acquired by the Sentinel-2 (S2) constellation. We formulate the S2 sharpening as an inverse problem and solve it using an unsupervised convolutional neural network (CNN), called S2UCNN. The proposed method extends the deep image prior provided by a CNN structure with S2 domain knowledge. We incorporate a modulation transfer function-based degradation model as a network layer. We add the 10 m bands to both the network input and output to take advantage of the multitask learning. Experimental results with a real S2 dataset show that the proposed method outperforms the competitive methods on reduced-resolution data and gives very high quality sharpened image on full-resolution data.

Sentinel-2 (S2) is a constellation of two satellites that frequently acquire optical imagery over land and coastal waters. The S2 sensors have three spatial resolutions: 10, 20, and 60 m. Many remote sensing applications require the spatial resolution to be at the highest resolution, i.e., 10 m for S2. To address this demand, researchers have proposed various methods that exploit the spectral and spatial correlation in multispectral data to sharpen the S2 bands to 10 m. In this paper, we fuse S2 data with high-resolution aerial RGB images. A method called S2Sharp is modified to include the red, green, and blue bands of the aerial image and sharpen S2 data to the resolution of the RGB image. The method, termed S2PF, is evaluated using an S2 image and aerial photographs of Reykjavik, Iceland.

This paper presents a wavelet-based block low-rank representations (WBBLRR) denoising method for hyperspectral images (HSIs). WBBLRR uses 3-D wavelet transformation to decompose HSI into different blocks, where each block utilizes a low-rank representations model to obtain the denoised block, and then uses inverse 3-D wavelet transformation for all the denoised blocks to obtain the denoised HSI. The proposed method is evaluated by using both simulated and real hyperspectral datasets.

This paper presents a non-local means low-rank approximation (NLMLRA) denoising method for hyperspectral images (HSIs). NLMLRA uses a Slanted Butterworth function to construct a low-rank approximation for non-local means (NLM) operator and is efficiently implemented based on Chebyshev polynomials. The proposed method is evaluated by using both simulated and real hyperspectral datasets.

Hyperspectral images (HSIs) are useful for many remote sensing applications. However, they are usually affected by noise that degrades the HSIs quality. Therefore, HSI denoising is important to improve the performance of subsequent HSI processing and analysis. In this article, we propose an HSI denoising method called Stein's unbiased risk estimate-convolutional neural network (SURE-CNN). The method is based on an unsupervised CNN and SURE. The main difference of SURE-CNN from existing supervised learning methods is that the SURE-based loss function can be computed only from noisy data. Since SURE is an unbiased estimate of the mean squared error (MSE) of an estimator, training a CNN using the SURE loss can yield similar results as using the MSE with ground truth in supervised learning. Also, a subspace version of SURE-CNN is proposed to reduce the running time. Extensive experimental results with both simulated and real data sets show that the SURE-CNN method outperforms the competitive methods in both objective and subjective assessments.

Blind hyperspectral unmixing is the process of expressing the measured spectrum of a pixel as a combination of a set of spectral signatures called endmembers and simultaneously determining their fractional abundances in the pixel. Most unmixing methods are strictly spectral and do not exploit the spatial structure of hyperspectral images (HSIs). In this article, we present a new spectral-spatial linear mixture model and an associated estimation method based on a convolutional neural network autoencoder unmixing (CNNAEU). The CNNAEU technique exploits the spatial and the spectral structure of HSIs both for endmember and abundance map estimation. As it works directly with patches of HSIs and does not use any pooling or upsampling layers, the spatial structure is preserved throughout and abundance maps are obtained as feature maps of a hidden convolutional layer. We compared the CNNAEU method to four conventional and three deep learning state-of-the-art unmixing methods using four real HSIs. Experimental results show that the proposed CNNAEU technique performs particularly well and consistently when it comes to endmembers' extraction and outperforms all the comparison methods.

Hyperspectral images (HSI) are composed of hundreds of spectral bands, covering a broad range of the electromagnetic spectrum. However, images can only be visualized using three spectral channels for red, green, and blue (RGB) colors. Generating realistic RGB images using HSI is seldom the main focus of remote sensing researchers, and is therefore sometimes lacking. In this paper, we present an algorithm which creates realistic color images of HSI, using standardized methods. Research, conducted on the human perception of color in the 1920s culminated in the CIE 1931 XYZ color space. The algorithm maps every spectral band in the visible spectrum to the XYZ color space, using D65 as the reference illuminant, and then maps the XYZ to the sRGB (standard Red Green Blue) color space. The image is gamma-corrected and finally thresholded to improve contrast. The method was validated using two HSIs, creating realistic color images.

This paper addresses the hyperspectral image (HSI) denoising problem by using Stein's unbiased risk estimate (SURE) based convolutional neural network (CNN). Conventional deep learning denoising approaches often use supervised methods that minimize a mean-squared error (MSE) by training on noisy-clean image pairs. In contrast, our proposed CNN-based denoiser is unsupervised and only makes use of noisy images. The method uses SURE, which is an unbiased estimator of the MSE, that does not require any information about the clean image. Therefore minimization of the SURE loss function can accurately estimate the clean image only from noisy observation. Experimental results on both simulated and real hyperspectral datasets show that our proposed method outperforms competitive HSI denoising methods.

Sentinel-2 (S2) satellite constellations can provide multispectral images of 10 m, 20 m, and 60 m resolution for visible, near-infrared (NIR) and short-wave infrared (SWIR) in the electromagnetic spectrum. In this paper, we present a sharpening method based on a symmetric skipped connection convolutional neural network, called SSC-CNN, to sharpen 20 m bands using 10 m bands. The main advantage of SSC-CNN architecture is that it brings the features of the input branch to the output, thus improving convergence without using too many deep layers. The proposed method uses the reduced-scale combination of 10 m bands and 20 m bands, and the observed 20 m bands as the training pairs. The experimental results using two Sentinel-2 datasets show that our method outperforms competitive methods in quantitative metrics and visualization.

This paper presents a local spatial-spectral correlation based mixtures of factor analyzers (LSSC-MFA) denoising method for hyperspectral image (HSI). HSIs are usually degraded by different noise types such as missing lines (ML), missing pixels (MP), salt and pepper noise (SP), and Gaussian noise. The proposed method, hierarchically, removes the mixed noise. Firstly, we develop a novel local spatial-spectral correlation (LSSC) method to remove the ML noise. Then LSSC-MFA uses the mixtures of factor analyzers (MFA) method to remove the MP, SP, and Gaussian noises. The performance of the proposed method has been validated using both real and simulated HSI datasets. Results on the simulated datasets confirm considerable improvements in terms of peak signal-to-noise ratio (PSNR) compared to the state-of-the-art denoising methods used in experiments. In addition, visual improvements can be observed in the case of real dataset experiments.

This paper presents two hyperspectral image (HSI) denoising methods, mixtures of factor analyzers (MFA) and wavelet-based MFA (WMFA). MFA uses a Gaussian mixture model to segment the original HSI into different parts, where each part follows Gaussian distribution and then utilizes a factor analyzer to get a low-rank factor loading matrix, and finally uses the inverse transformation of the matrix to get the denoised hyperspectral dataset. WMFA uses the MFA in the wavelet domain to remove the noise in HSI. The proposed methods are evaluated by using both simulated and real hyperspectral datasets.

Observation of the Earth using satellites mounted with optical sensors is an important application of remote sensing. Owing to physical constraints, multispectral (MS) sensors acquire images of lower spatial resolution than a single-band panchromatic (PAN) sensor that acquires images of the same scene. Pansharpening fuses the MS and PAN images to obtain an MS image with the same spatial resolution as the PAN image. In this letter, we propose to expand a method, initially developed for Sentinel-2 single-sensor sharpening, for pansharpening. The expanded method is based on solving a non-convex MS acquisition model using optimization methods based on cyclic decent and manifold optimization. The tuning parameters of the method are chosen using Bayesian optimization with reduced-scale evaluation. The proposed method is compared with a number of established pansharpening methods and is validated using both synthetic and real data sets.

Answering to metrological constraints typically required in the context of industrial and medical applications, a spectral difference space is introduced in this work. In this space, an acquired hyperspectral data is treated as measurements. Then, modelling the spectral difference space as multivariate Normal laws, a Gaussian mixture model is used in a classification task of remote sensing images. An encouraging result is obtained, comparing the proposed space with a data-driven one. Moreover, it offers a starting point in developing a directly interpretable spectral analysis tools.

This paper introduces a novel deep learning based method for blind hyperspectral unmixing. The technique utilizes multi-task learning (MTL) through multiple parallel autoencoders to simultaneously unmix a neighborhood of pixels at a time. The sharing of hidden representations between autoencoders enables the method to take advantage of spatial correlations in the hyperspectral image (HSI). The method is evaluated using real HSI data and compared to three state-of-the-art techniques. The proposed method outperforms all the comparison methods in the experiments.

The fusion of a low spatial resolution multispectral image and a high spatial resolution panchromatic image, i.e., pan-sharpening is an important technique in remote sensing where high resolution imagery is needed. Two of the largest families of such methods are the component substitution (CS) and multi-resolution analysis (MRA) methods. These families of methods can be described by general detail injection schemes which are closely related. In this paper, we propose pansharpening methods which are based on directly implementing these schemes using a convolutional neural network (CNN) such that the mean squared error between the down-sampled fused image and the observed multispectral image is minimized. Using a simulated Pleiades dataset we demonstrate that the proposed approach gives excellent results when compared to other state-of-the-art CS, MRA and CNN methods.

In this paper, we present a deep learning based method for blind hyperspectral unmixing in the form of a fully convolutional autoencoder. The technique is the first to fully utilize the spatial structure of hyperspectral images (HSIs) for both endmember and abundance map estimation. The framework has many advantages over older methods as it works directly with patches of HSIs' and thus preserves the spatial structure while abundance maps arise naturally as feature maps of a hidden convolutional layer. We evaluate the proposed method using a real HSI and compare it to three state-of-the-art methods. The proposed method outperforms all the comparison methods in the experiments.

Blind hyperspectral unmixing is the process of decomposing hyperspectral images (HSIs) into pure material spectra (endmembers) and abundances. In this paper, we examine scaling the pixels of the HSI inversely proportional to their ℓ2norm, controlled with a tuning parameter. We promote sparse abundances using an ℓqpenalty and softly enforce the abundance sum constraint using matrix augmentation. The minimization problem is solved using a variant of sparse nonnegative matrix factorization (NMF) and all tuning parameters are selected using Bayesian optimization. The proposed method is evaluated using two real hyperspectral images.

This paper presents four dimensionality reduction methods, supervised mixtures of factor analyzers (SMFA), semi-supervised mixtures of factor analyzers (S2MFA), supervised deep mixtures of factor analyzers (SDMFA) and semi-supervised deep mixtures of factor analyzers (S2DMFA), for hyperspectral image (HSI) classification. The performance of SMFA, S2MFA, SDMFA, and S2DMFA dimensionality reduction methods for classification using real HSI is evaluated in this paper. Experimental results are compared to more conventional methods like probabilistic principal component analysis, factor analysis, mixtures of factor analyzers and deep mixtures of factor analyzers and it is shown that the proposed methods give better results.

This paper presents two dimensionality reduction methods, mixtures of factor analyzers (MFA) and deep mixtures of factor analyzers (DMFA), for classification of hyperspectral image (HSI). DMFA consists of two layers of MFA and can extract more information from HSI than MFA can. The performance of MFA and DMFA dimensionality reduction methods for classification using real HSI is evaluated in this paper. Experimental results are compared to conventional methods like probabilistic principal component analysis and factor analysis and it is shown that MFA and DMFA give better results.

Recently, the Sentinel-2 (S2) satellite constellation was deployed for mapping and monitoring the Earth environment. Images acquired by the sensors mounted on the S2 platforms have three levels of spatial resolution: 10, 20, and 60 m. In many remote sensing applications, the availability of images at the highest spatial resolution (i.e., 10 m for S2) is often desirable. This can be achieved by generating a synthetic high-resolution image through data fusion. To this end, researchers have proposed techniques exploiting the spectral/spatial correlation inherent in multispectral data to sharpen the lower resolution S2 bands to 10 m. In this paper, we propose a novel method that formulates the sharpening process as a solution to an inverse problem. We develop a cyclic descent algorithm called S2Sharp and an associated tuning parameter selection algorithm based on generalized cross validation and Bayesian optimization. The tuning parameter selection method is evaluated on a simulated data set. The effectiveness of S2Sharp is assessed experimentally by comparisons to state-of-the-art methods using both simulated and real data sets.

In this paper, a hyperspectral feature extraction (FE) method called sparse and smooth low-rank analysis (SSLRA) is proposed. First, we propose a new low-rank model for hyperspectral images (HSIs). In the new model, HSI is decomposed into smooth and sparse unknown features which live in an unknown orthogonal subspace. Then, the sparse and smooth features are simultaneously estimated using a non-convex constrained penalized cost function. In the experiments' SSLRA is applied on a real HSI and the smooth features extracted are used for the HSI classification. The results confirm improvements in classification accuracies compared to state-of-the-art FE methods.

Hyperspectral unmixing consists of estimating pure material spectra (endmembers) and their corresponding abundances in hyperspectral images. In this paper, a blind nonlinear hyperspectral unmixing algorithm is presented. The algorithm promotes sparse abundance maps using an lq regularizer and assumes that the spectra are mixed according to an extension to generalized bilinear model, called the Fan model. The algorithm is evaluated using both simulated and real hyperspectral data.

Multiresolution optical remote sensing systems often have a spatial resolution that varies between bands. An example is the Sentinel-2 (S2) constellation which has three levels of spatial resolution 10m, 20m, and 60m. Recently, researchers have exploited the spectral/spatial correlation inherent in multispectral data to sharpen the lower resolution S2 bands. In this paper, we propose a low-rank method that formulates the sharpening process as a solution to a cost function. We develop an iterative algorithm based on cyclic descent and call it S2Sharp-CD. We evaluate the method on a simulated dataset and compare it to a state-of-the-art approach.

Blind hyperspectral unmixing (HU) is the task of jointly estimating the spectral signatures of materials and abundances in hyperspectral images. Most unmixing algorithms assume the linear mixture model, however, nonlinear models have recently gained interest, as they represent more complicated scenes. This letter proposes two blind nonlinear HU algorithms. The former algorithm assumes that the spectra are mixed according to the generalized bilinear model, while the latter assumes an extension to this model, called the Fan model. Both the algorithms use the ℓqregularizer to promote sparse abundances and solve the minimization problems using cyclic descent. The algorithms are evaluated and compared with other unmixing algorithms using both simulated and real data.

Hyperspectral unmixing is a challenging inverse problem that involves determining the fractional abundances of the representive material (endmembers) in each pixel. In this paper, we develop a neural network autoencoder, that dynamically exploits the sparsity of the abundances and enforces the abundance sum constraint (ASC) for hyperspectral unmixing. Instead of using the conventional mean square error (MSE) objective function, we use the spectral information divergence (SID) measure. Experiments are performed using a real hyperspectral dataset and we compare results obtained using both MSE and SID. It is demonstrated by qualitative inspection that using SID gives significantly better results than using MSE.

In this paper, we present a fast blind multitemporal hyperspectral unmixing algorithm, using an l1penalty to promote sparse abundances. The method is able to account for different acquisition conditions of multitemporal images, by allowing the spectral signatures in the different temporal images to vary. The new algorithm is tested on simulated data and applied on real hyperspectral data.

In this paper, hyperspectral data is modeled as a combination of a sparse component, a low rank component and noise. The low rank component is a product of the endmembers and the abundances in an image, and the sparse component is composed of outliers and structured noise. Outliers and structured noise in this context are, e.g. band specific noise, vertical or horizontal artifacts or saturated pixels. Sparse and low rank matrix decomposition (SLR) is a method that has recently been developed for estimating those components. Here, an algorithm based on ℓ1SLR is developed using sparse blind hyperspectral unmixing and soft thresholding. The number of endmembers and the sparsity parameters are selected using the extended Bayesian information criterion (EBIC). The proposed algorithm is evaluated using a real remote sensing hyperspectral image.

In this letter, we propose a method using a 3-D convolutional neural network to fuse together multispectral and hyperspectral (HS) images to obtain a high resolution HS image. Dimensionality reduction of the HS image is performed prior to fusion in order to significantly reduce the computational time and make the method more robust to noise. Experiments are performed on a data set simulated using a real HS image. The results obtained show that the proposed approach is very promising when compared with conventional methods. This is especially true when the HS image is corrupted by additive noise.

Hyperspectral restoration is a preprocessing step for hyperspectral imagery. In this letter, we propose a parameter-free method for the restoration of hyperspectral images (HSIs) called HyRes. The restoration method is based on a sparse low-rank model that uses the ℓ1penalized least squares for estimating the unknown signal. The Stein's unbiased risk estimator is exploited to select all the parameters of the model yielding a fully automatic (parameter free) technique. Experimental results confirm that HyRes outperforms the state-of-the-art techniques in terms of signal-to-noise ratio, structural similarity index, and spectral angle distance for a simulated data set and in terms of noise-level estimation for the real data sets used in this letter. In the experiments, it was noted that HyRes is computationally less expensive compared with competitive techniques. Therefore, HyRes can be used as a reliable automatic preprocessing step for further analysis of HSIs.

Blind hyperspectral unmixing jointly estimates spectral signatures and abundances in hyperspectral images (HSIs). Hyperspectral unmixing is a powerful tool for analyzing hyperspectral data. However, the usual huge size of HSIs may raise difficulties for classical unmixing algorithms, namely, due to limitations of the hardware used. Therefore, some researchers have considered distributed algorithms. In this paper, we develop a distributed hyperspectral unmixing algorithm that uses the alternating direction method of multipliers and ℓ1sparse regularization. The hyperspectral unmixing problem is split into a number of smaller subproblems that are individually solved, and then the solutions are combined. A key feature of the proposed algorithm is that each subproblem does not need to have access to the whole HSI. The algorithm may also be applied to multitemporal HSIs with due adaptations accounting for variability that often appears in multitemporal images. The effectiveness of the proposed algorithm is evaluated using both simulated data and real HSIs.

There has been significant interest in sparse inverse covariance estimation in areas such as statistics, machine learning, and signal processing. In this problem, the sparse inverse of a covariance matrix of a multivariate normal distribution is estimated. A Penalised Log-Likelihood (PLL) optimisation problem is solved to obtain the matrix estimator, where the penalty is responsible for inducing sparsity. The most natural sparsity promoting penalty is the non-convex l0function. Due to speed and memory limitations, the existing algorithms for dealing with the non-convex l0PLL problem are unable to be used in high dimensional settings. Here we address this issue by presenting a new block iterative approach for this problem, which can handle large-scale data sizes. Simulations demonstrate that our approach outperforms existing methods for this problem.

Non-negative matrix factorization (NMF) has found use in fields such as remote sensing and computer vision where the signals of interest are usually non-negative. Data dimensions in these applications can be huge and traditional algorithms break down due to unachievable memory demands. One is then compelled to consider distributed algorithms. In this paper, we develop for the first time a distributed version of NMF using the alternating direction method of multipliers (ADMM) algorithm and dyadic cyclic descent. The algorithm is compared to well established variants of NMF using simulated data, and is also evaluated using real remote sensing hyperspectral data.

Blind hyperspectral unmixing is the task of jointly estimating the spectral signatures of material in a hyperspectral images and their abundances at each pixel. The size of hyperspectral images are usually very large, which may raise difficulties for classical optimization algorithms, due to limited memory of the hardware used. One solution to this problem is to consider distributed algorithms. In this paper, we develop a distributed sparse hyperspectral unmixing algorithm using the alternating direction method of multipliers (ADMM) algorithm and ℓ1sparse regularization. Each sub-problem does not need to have access to the whole hyperspectral image. The algorithm is evaluated using a very large real hyperspectral image.

Big data applications, such as medical imaging and genetics, typically generate datasets that consist of few observations n on many more variables p, a scenario that we denote asp ≫ n. Traditional data processing methods are often insufficient for extracting information out of big data. This calls for the development of new algorithms that can deal with the size, complexity, and the special structure of such datasets. In this paper, we consider the problem of classifying p ≫ n data and propose a classification method based on linear discriminant analysis (LDA). Traditional LDA depends on the covariance estimate of the data, but when p ≫ n, the sample covariance estimate is singular. The proposed method estimates the covariance by using a sparse version of noisy principal component analysis (nPCA). The use of sparsity in this setting aims at automatically selecting variables that are relevant for classification. In experiments, the new method is compared to state-of-the art methods for big data problems using both simulated datasets and imaging genetics datasets.

Pansharpening is the process of fusing a high-resolution panchromatic image and a low-spatial-resolution multispectral image to yield a high-spatial-resolution multispectral image. This is a typical ill-posed inverse problem, and in the past two decades, many methods have been proposed to solve it. Still, there is no general consensus on the best way to quantitatively evaluate the spectral and spatial quality of the fused image. In this paper, we compare the two most widely used and accepted methods for quality evaluation. The first method is the verification of the synthesis property which states that the fused image should be as identical as possible to the multispectral image that the sensor would observe at a higher resolution. This is impossible to verify unless the observed images are spatially degraded so that the original observed multispectral image can be used as reference. The second method is to use metrics that do not use a reference, such as the quality no reference (QNR) metrics. However, there is another property, i.e., the consistency property, which states that the fused image reduced to the resolution of the original multispectral image should be as identical to the original image as possible. This has generally been considered a necessary condition that does not have to imply correct fusion. Using real WorldView-2 and QuickBird data and a total of 18 component substitution and multiresolution analysis methods, we demonstrate that the consistency property can indeed be used to give reliable assessment of the relative performance of pansharpening methods and is superior to using the QNR metrics.

In this paper, a novel feature extraction method, called orthogonal total variation component analysis (OTVCA), is proposed for remotely sensed hyperspectral data. The features are extracted by minimizing a total variation (TV) penalized optimization problem. The TV penalty promotes piecewise smoothness of the extracted features which is useful for classification. A cyclic descent algorithm called OTVCA-CD is proposed for solving the minimization problem. In the experiments, OTVCA is applied on a rural hyperspectral image having low spatial resolution and an urban hyperspectral image having high spatial resolution. The features extracted by OTVCA show considerable improvements in terms of classification accuracy compared with features extracted by other state-of-the-art methods.

Blind hyperspectral unmixing involves jointly estimating endmembers and fractional abundances in hyperspectral images. An endmember is the spectral signature of a specific material in an image, while an abundance map specifies the amount of a material seen in each pixel in an image. In this paper, a new cyclic descent algorithm for blind hyperspectral unmixing using total variation (TV) and ℓq sparse regularization is proposed. Abundance maps are both spatially smooth and sparse. Their sparsity derives from the fact that each material in the image is not represented in all pixels. The abundance maps are assumed to be piecewise smooth since adjacent pixels in natural images tend to be composed of similar material. The TV regularizer is used to encourage piecewise smooth images, and the ℓqregularizer promotes sparsity. The dyadic expansion decouples the problem, making a cyclic descent procedure possible, where one abundance map is estimated, followed by the estimation of one endmember. A novel debiasing technique is also employed to reduce the bias of the algorithm. The algorithm is evaluated using both simulated and real hyperspectral images.

Significant attention has been given to minimizing a penalized least squares criterion for estimating sparse solutions to large linear systems of equations. The penalty induces sparsity and the natural choice is the so-called l0norm. In this paper we develop a Momentumized Iterative Shrinkage Thresholding (MIST) algorithm for minimizing the resulting non-convex criterion and prove its convergence to a local minimizer. Simulations on large data sets show superior performance of the proposed method to other methods.

High dimensional data is often modeled as a linear combination of a sparse component, a low-rank component, and noise. An example is a video sequence of a busy scene where the background is the low-rank part and the foreground, e.g. moving pedestrians, is the sparse part. Sparse and low rank (SLR) matrix decomposition is a recent method that estimates those components. In this paper we develop an l0based SLR method and an associated tuning parameter selection method based on the extended Bayesian information criterion (EBIC) method. In simulations the new algorithm is compared with state of the art algorithms from the literature.

In the past two decades, many methods have been proposed to fuse low resolution multispectral (MS) and high resolution panchromatic (Pan) images, i.e., pansharpening. Two large families of such methods are component substitution (CS) and multiresolution analysis methods (MRA). We develop a model based method for pansharpening based on minimizing a cost function which includes a data fidelity term, a detail injection term and a total variation (TV) term. The model takes into account the modulation transfer function (MTF) and spectral response of the sensor. The resulting iterative method not only sharpens the MS image with details from the Pan image but is also able to extract important information from the MS image itself via MTF-based deconvolution. We compare the proposed method to a number of state-of-the-art CS and MRA pansharpening methods using a real WorldView-2 dataset and show that it gives excellent results with details that all the CS and MRA methods can not extract.

The fusion of low resolution multispectral (MS) images and high resolution panchromatic (PAN) images, i.e., pansharpening, is an important technique in remote sensing and has many applications where high resolution imagery is important. Component substitution (CS) and multiresolution analysis (MRA) are two large families of pansharpening methods that are fast and computationally efficient. They can be described using a general framework, where details from the PAN image are added to the upsampled and interpolated MS image. However, these methods often suffer from spectral and spatial distortions. We propose a pre-processing step, where instead of just interpolating the MS image to the resolution scale of the PAN image, we do a deconvolution of the interpolated MS image based on the sensor's modulation transfer function (MTF). This results in large improvement gains in the spectral and spatial quality of the fused image. We demonstrate our method using a real WorldView-2 dataset and show that our approach significantly improves the tested methods in both the CS and MRA families of pansharpening methods.

Blind hyperspectral unmixing jointly estimates both the endmembers and the abundances of hyperspectral images. The endmembers represent the spectral signatures of material found in the image and the abundances specify the amount of each material seen in each pixel in the image. In this paper, a blind hyperspectral unmixing method for feature extraction and classification using total variation (TV) and ℓqsparse regularization is proposed. The abundances found are used as features for classification. The classification results are compared to results obtained using Principal Component analysis (PCA) and also to results obtained using hyperspectral unmixing using only TV and sparsity, respectively.

The identification of the signal subspace is a very important first step for most hyperspectral algorithms. In this letter, we investigate the important problem of identifying the hyperspectral signal subspace by minimizing the mean squared error (MSE) between the true signal and an estimate of the signal. Since it is dependent on the true signal, the MSE is uncomputable in practice, and so we propose a method based on Stein's unbiased risk estimator that provides an unbiased estimate of the MSE. The resulting method is simple and fully automatic, and we evaluate it using both simulated and real hyperspectral data sets. Experimental results show that our proposed method compares well to recent state-of-the-art subspace identification methods.

Principal component analysis (PCA) is one of the most widely used methods in multivariate signal processing. An important problem is to select the number of principal components (PCs). In this paper we develop an automatic method for selecting the number of PCs based on Stein's unbiased risk estimator (SURE). In simulations the new method outperforms state of the art cross-validation methods.

In remote sensing, due to cost and complexity issues, multispectral (MS) and hyperspectral (HS) sensors have significantly lower spatial resolution than panchromatic (PAN) images. Recently, the problem of fusing coregistered MS and HS images has gained some attention. In this paper, we propose a novel method for fusion of MS/HS and PAN images and of MS and HS images. MS and, more so, HS images contain spectral redundancy, which makes the dimensionality reduction of the data via principal component (PC) analysis very effective. The fusion is performed in the lower dimensional PC subspace; thus, we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and very high tolerance to noise in the observed data. Examples are presented using WorldView 2 data and a simulated data set based on a real HS image, with and without added noise.

There are two main approaches to independent component analysis (ICA); maximization of non-Gaussianity of the sources and the exploitation of temporal correlation in Gaussian sources. In this paper, we present a novel sparse noisy ICA model where we have introduced temporal correlation in the sources, described by a first order auto regressive (AR(1)) process. The correlation structure of the sources eliminates the rotational invariance of the estimates, enabling their separation. Using simulated data, we demonstrate both source separation and denoising, where we compare our results to a sparse PCA method and the fastICA method. Additionally, we apply the method on a real hyperspectral dataset.

Sparse component analysis (SCA) is a widely used method for solving the blind source separation problem. We develop a new cyclic descent algorithm for SCA based on a dyadic expansion. To select the associated tuning parameter a method based on the Bayesian information criterion is developed. In simulations the new algorithm is compared with state of the art algorithms from the literature.

Due to cost and complexity issues, hyperspectral (HS) images have lower spatial resolution than multispectral (MS) and panchromatic (PAN) images. We present a novel method for fusing both MS and PAN images and also HS and MS images, based on their statistical properties in the wavelet domain. HS images contain spectral redundancy that makes the dimensionality reduction of the data via principal component analysis (PCA) very effective. The fusion is performed in the lower dimensional PC subspace so we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and a very high tolerance to noise in the observed data. Examples are presented using World View 2 data and also a simulated dataset based on a real HS image, with and without noise.

Hyperspectral images (HSI) are often corrupted by noise making their analysis and interpretation difficult. In this paper we develop a sparse low rank model for HSI, which is useful for denoising. The two key benefits of the model for denoising are dimensionality reduction via noisy principal component analysis (nPCA) and the exploitation of sparse-ness in the dual-tree complex wavelet transform (CWT) coefficients of the loading matrix associated with the principal components (PCs). We present denoising examples of both synthetic and real data and compare our method to a PCA based 2-dimensional (2D) bivariate shrinkage method.

In this paper, a hyperspectral feature extraction method is proposed. A low-rank linear model using the right eigenvector of the observed data is given for hyperspectral images. A total variation (TV) based regularization called Low-Rank TV regularization (LRTV) is used for hyperspectral feature extraction. The feature extraction is used for hyperspectral image classification. The classification accuracies obtained are significantly better than the ones obtained using features extracted by Principal Component Analysis (PCA) and Maximum Noise Fraction (MNF).

Mean squared error (MSE) is commonly used for evaluating the performance of hyperspectral imaging (HSI) methods. MSE depends on the true (unknown) signal to be estimated and is therefore not computable for real data. Therefore, HSI methods are usually evaluated using simulated data. Stein's unbiased risk estimator (SURE) is an unbiased estimator of the MSE that does not require knowledge of the true signal. The main aim of this paper is to promote the use of SURE for evaluating HSI models. To achieve that goal we compare three wavelet models, spectral, spatial and spectral-spatial, for hyperspectral images. Hyperspectral images are modeled based on their sparse wavelet components. The penalized least squares with i.e. penalty (to promote sparsity) is considered for sparse reconstruction. By comparing the SURE values for the three models, it is shown that the spatial model performs better than spectral model and spectral-spatial model outperforms both spectral and spatial models.

In this paper, an effective method is proposed that combines supervised and unsupervised unmixing. We assume a linear model for the hyperspectral data and incorporate information about endmembers that are known to be in the data into the model. This information can be acquired from a spectral library or extracted from the data. Utilizing a priori information can both improve the unmixing, and reduce the complexity of the problem. The method is quantitatively evaluated using simulated data and it is shown that the unmixing results improve and the computational time decreases when a priori information is used. The method is also applied on a real hyperspectral data set of an urban landscape. The estimated abundance maps improve when information about known endmembers is incorporated into the model.

In this letter, we present a new method for the pansharpening of multispectral satellite imagery. Pansharpening is the process of synthesizing a high spatial resolution multispectral image from a low spatial resolution multispectral image and a high-resolution panchromatic (PAN) image. The method uses total variation to regularize an ill-posed problem dictated by a widely used explicit image formation model. This model is based on the assumptions that a linear combination of the bands of the pansharpened image gives the PAN image and that a decimation of the pansharpened image gives the original multispectral image. Experimental results are based on two real datasets and the quantitative quality of the pansharpened images is evaluated using a number of spatial and spectral metrics, some of which have been recently proposed and do not need a reference image. The proposed method compares favorably to other well-known methods for pansharpening and produces images of excellent spatial and spectral quality.

Hyperspectral remote sensing technology allows one to acquire a sequence of possibly hundreds of contiguous spectral images from ultraviolet to infrared. Conventional spectral classifiers treat hyperspectral images as a list of spectral measurements and do not consider spatial dependences, which leads to a dramatic decrease in classification accuracies. In this paper, a new automatic framework for the classification of hyperspectral images is proposed. The new method is based on combining hidden Markov random field segmentation with support vector machine (SVM) classifier. In order to preserve edges in the final classification map, a gradient step is taken into account. Experiments confirm that the new spectral and spatial classification approach is able to improve results significantly in terms of classification accuracies compared to the standard SVM method and also outperforms other studied methods.

In this paper, a method called wavelet-based sparse reduced-rank regression (WSRRR) is proposed for hyperspectral image restoration. The method is based on minimizing a sparse regularization problem subject to an orthogonality constraint. A cyclic descent-type algorithm is derived for solving the minimization problem. For selecting the tuning parameters, we propose a method based on Stein's unbiased risk estimation. It is shown that the hyperspectral image can be restored using a few sparse components. The method is evaluated using signal-to-noise ratio and spectral angle distance for a simulated noisy data set and by classification accuracies for a real data set. Two different classifiers, namely, support vector machines and random forest, are used in this paper. The method is compared to other restoration methods, and it is shown that WSRRR outperforms them for the simulated noisy data set. It is also shown in the experiments on a real data set that WSRRR not only effectively removes noise but also maintains more fine features compared to other methods used. WSRRR also gives higher classification accuracies.

Hyperspectral unmixing is an important technique for analyzing remote sensing images. In this paper, we consider and examine the ℓq, 0 ≤ q ≤ 1 penalty on the abundances for promoting sparse unmixing of hyperspectral data. We also apply a first-order roughness penalty to promote piecewise smooth end-members. A novel iterative algorithm for simultaneously estimating the end-members and the abundances is developed and tested both on simulated and two real hyperspectral data sets. We present an extensive simulation study where we vary both the SNR and the sparsity of the simulated data and identify the model parameters that minimize the reconstruction errors and the spectral angle distance. We show that choosing 0 ≤ q1penalty when the SNR is low or the sparsity of the underlying model is high. We also examine the effects of the imposing the abundance sum constraint using a real hyperspectral data set.

Finding low rank nonnegative decomposition of multivariate data has many important applications in signal processing. A standard method is the nonnegative matrix factorization (NMF). In recent years, many algorithm have been proposed for NMF. However, an important problem that has not received as much attention is the selection of the rank of NMF. In this paper we develop a method for selecting the rank of NMF based on the Stein's unbiased risk estimator (SURE). In simulations we compare the method against crossvalidation. In addition we apply the method for selecting the rank of NMF for high dimensional hyperspectral data.

In this work, a new framework for accurate classification of hyperspectral images is proposed. The new method is based on Hidden Markov Random Field and its Expectation Maximization (HMRF-EM) and Support Vector Machine (SVM) classifier. In order to preserve edges in final map, the Sobel edge detector is used. Result confirms that the combination of the spectral and spatial information can significantly improve results compared to the standard SVM method.

In this paper we consider pansharpening of multispectral satellite imagery based on solving an under-determined inverse problem regularized by the ℓ1-norm of the coefficients of overcomplete multi-scale transforms which all are tight-frame systems. There are two main approaches in sparsity promoting ℓ1-norm regularization, the analysis and the synthesis approach. We perform a number of experiments using two real and well known datasets where the focus is the comparison of the two approaches. One dataset includes a high resolution reference image while the other needs to be degraded prior to pansharpening in order to use the original multispectral image as the reference. Experiments are performed for a range of values for the regularization parameter, where each resulting pansharpened image is evaluated using three quality metrics. The behavior of those metrics as a function of the regularization parameter is compared for the analysis and synthesis formulations and it is shown that analysis gives better results.

This paper deals with hyperspectral image reconstruction using a new linear model and Sparse Regularization (SR). The new model is based on Principal Components (PCs) and wavelets. Since the hyperspectral PCs are not spatially sparse, wavelet is applied to get spatially sparse representation. Sparse regularization is used to recover the corrupted signal. The regularization parameter is chosen by Stein's Unbiased Risk Estimator (SURE). The results show improvements for simulated data sets compare to other denoising methods based on Signal to Noise Ratio (SNR). In addition, the methods are applied on a real noisy data set, and the results of the new method demonstrate visual improvement. The proposed approach is automatic, fast and has the ability to be applied on very large data sets.

We propose CUR methods for hyperspectral unmixing that decompose the data matrix into non-negative endmembers and abundance maps. The endmembers will be selected from a dictionary constructed from the data matrix. Each endmember will coincide with certain columns of the data matrix. By doing this we are assured that the dictionary will be physically meaningful and may be interpreted unambiguously from the data set. This assumption, that the endmembers are contained within the data, is called the pixel purity assumption. We compare two regularization terms to promote sparsity in our solutions, the first is ℓ2regularization and the second is vector ℓ0regularization. The methods are evaluated both on simulated data and a real hyperspectral image of an urban landscape.

Hyperspectral unmixing is the task of decomposing hyperspectral images into endmembers and their abundances. The endmembers are spectral signatures of specific material in the image and the abundances dictate the amount of the material found in each pixel. In this paper we present a blind signal separation method, based on the total variation penalty, that simultaneously estimates the endmembers and the abundances. We evaluate our method using both simulated and a real data set.

Multivariate regression is one of the most widely applied multivariate statistical methods with many uses across a range of disciplines. But the number of parameters increases exponentially with dimension and reduced-rank regression (RRR) is a well known approach to dimension reduction. But traditional RRR applies only to an overdetermined system. For increasingly common undetermined systems this issue can be managed by regularization, e.g., with a quadratic penalty. A significant problem is then the choice of the two tuning parameters: one discrete i.e., the rank; the other continuous i.e., the Tikhonov penalty parameter. In this paper we resolve this problem via Stein's unbiased risk estimator (SURE). We compare SURE to cross-validation and apply it on both simulated and real data sets.

In this paper we present a novel model based sparse principal component analysis method based on the l0penalty. We develop an estimation method based on the generalized EM algorithm and iterative hard thresholding and an associated model selection method based on Bayesian information criterion (BIC). The method is compared to a previous sparse PCA method using both simulated data and DNA microarray data.

Images obtained using Synthetic Aperture Radar (SAR) are corrupted by speckle. Speckle noise results from the chaotic interference of backscattered electromagnetic waves and makes the analysis, interpretation and classification of SAR images difficult. In this paper, we present a denoising algorithm based on Total Variation (TV) regularization. While this kind of denoising algorithm is not new, we propose to select the regularization parameter by minimizing the estimate of the mean square error (MSE) between the denoised image and the clean image. We do not have to know the clean image because we use a statistically unbiased MSE estimate - Stein's Unbiased Risk Estimate (SURE), that depends on the observed image and the estimated image. However, since it is difficult to derive SURE analytically for this kind of problem, we estimate SURE using stochastic methods. We present results using both a simulated image and real SAR image.

In this paper we present a new method for the pansharpening of multi-spectral satellite imagery. This method is based on a simple explicit image formation model which leads to an ill posed problem that needs to be regularized for best results. We use both Tikhonov (ridge regression) and Total Variation (TV) regularization. We develop the solutions to these two problems and then we address the problem of selecting the optimal regularization parameter λ. We find the value of λ that minimizes Stein's unbiased risk estimate (SURE). For ridge regression this leads to an analytical expression for SURE while for the TV regularized solution we use Monte Carlo SURE where the estimate is obtained by stochastic means. Finally, we present experiment results where we use quality metrics to evaluate the spectral and spatial quality of the resulting pansharpened image.

In this paper, we propose a denoising method for hyperspectral images using 3D wavelets. We use the sparse analysis regularization using a 3D overcomplete wavelet dictionary. The minimization problem is solved using iterative Chambolle algorithm. The simulation results show that the 3D dictionary outperforms the 2D one, in terms of Peak Signal to Noise Ratio (PSNR). Denosing hysperspectral cubes is likely to increase the classification accuracy of the hyperspectral data since it can enhance the spectral profiles (or features) that can be useful to discriminate between information classes.

Hyperspectral unmixing is the process where the reflectance spectrum from a mixed pixel is decomposed into separate distinct spectral signatures (endmembers). A mixed pixel results when spectra from more than one material is recorded by a sensor in one pixel. The goal of linear unmixing is to identify the number of endmembers in an image, the endmembers themselves and their abundances in each pixel. This paper presents a new smooth method for unmixing hyperspectral images using nonnegative cyclic descent. The proposed method uses iterative cyclic descent algorithm to find the endmembers and their abundances. The algorithm uses an ℒ1norm to promote sparseness in the abundances. Because the spectrum of the endmembers varies smoothly, a first order roughness penalty is added to discourage roughness in the endmembers. The algorithm does not use any prior information about the data. The method is tested using a real hyperspectral image of an urban landscape.

Reduced rank regression (RRR) has found application in various fields of signal processing. In this paper we propose a novel extension of the RRR model which we call sparse variable reduced rank regression (svRRR). By using a vector l1penalty we remove variables completely from the RRR. The proposed estimation algorithm involves optimization on the Stiefel manifold and we illustrate it both on a simulated and a real functional magnetic resonance imaging (fMRI) data set.

The group Lasso is an extension of the Lasso or l1-penalised least squares procedure. It forces simultaneous zeroing of groups of variables and has already been applied to sparse component analysis and ill-conditioned inverse problems. In this paper we address the unresolved problem of threshold or penalty parameter selection.

Functional Magnetic Resonance (fMRI) data is most often analyzed using linear regression type methods that consider each voxel separately or by using exploratory methods such as Principal Component Analysis (PCA) or Independent Component Analysis (ICA). In this paper we introduce a model, which we call XnPCA, that combines regression with PCA. Unlike the linear regression methods XnPCA allows for non-stationary noise. Additionally, since XnPCA is based on the maximum likelihood framework the Bayesian information criterion (BIC) can be used for model selection and comparison. We compare XnPCA to a regression model commonly used in fMRI research using real data from a combined visual-motor experiment.

Sparse principal component analysis combines the idea of sparsity with principal component analysis (PCA). There are two kinds of sparse PCA; sparse loading PCA (slPCA) which keeps all the variables but zeroes out some of their loadings; and sparse variable PCA (svPCA) which removes whole variables by simultaneously zeroing out all the loadings on some variables. In this paper we propose a model based svPCA method based on the l0penalty. We compare the detection performance of the proposed method with other subset selection method using a simulated data set. Additionally, we apply the method on a real high dimensional functional magnetic resonance imaging (fMRI) data set.

International audience

Recently there has developed considerable interest in using sparseness with PCA. Almost all previous methods concentrate on zeroing out some loadings. Here we develop a new approach which zeros out whole variables automatically. We formulate a vector l1penalized PCA criterion and optimize it by steepest descent along geodesic on a Grassman manifold. This ensures that each step obeys PCA orthogonality as well as an invariance property of the criterion. We show in simulations that it outperforms a previous svPCA algorithm and apply it to a real high dimensional functional Magnetic Resonance Imaging (fMRI) data.

Synthetic aperture radar (SAR) images are corrupted by speckle noise due to random interference of electromagnetic waves. The speckle degrades the quality of the images and makes interpretation, analysis and classification of SAR images harder. Therefore, some speckle reduction is necessary prior to the processing of SAR images. The speckle noise can be modeled as multiplicative i.i.d. Rayleigh noise. Sveinsson and Benediktsson [1996], proposed an adaptive sigmoid thresholding method for SAR images in the wavelet domain. The coefficients thresholding for this method is based on the choice of parameters in the sigmoid thresholding function. They were chosen according to a visual appreciation, i.e., by anad hocmethod. We propose to select these parameters by minimizing an estimate of square error between the clean image and the denoised one. The key point is that we have in our proposal computable, statistically unbiased, MSE estimate - Stein's Unbiased Risk Estimate (SURE) - that depends on the noisy image alone, not on the clean image. We apply the proposed method on an SAR images, both simulated and real data.

Principal component analysis (PCA) is probably the best known method for dimensionality reduction. Perhaps the most important problem in PCA is to determine the number of principal components in a given data set, and in effect separate signal from noise in the data set. Many methods have been proposed to deal with this problem but almost all of them fail in the important practical case when the number of observations is comparable to the number of variables, i.e., the realm of random matrix theory (RMT). In this paper, we propose to use Stein's unbiased risk estimator (SURE) to estimate, with some assistance from RMT, the number of principal components. The method is applied on simulated data and compared to BIC and the Laplace method.

Multivariate methods such as principal component analysis (PCA) and independent component analysis (ICA) have been found to be useful in functional magnetic resonance imaging (fMRI) research. They are often able to decompose the fMRI data so that the researcher can associate their components to some biological processes of interest such as the brain response resulting from a stimulus. In this paper we develop a new smooth version of the PCA derived from a maximum likelihood framework. We are thus led to an unusual use of AIC, BIC namely to choose two (rather than one) parameters simultaneously; the number of principal components and the degree of smoothness. The algorithm is applied to real fMRI data

PCA has found use as an exploratory technique for fMRI analysis. However underlying it is an implicit model that while allowing temporal non-stationary covariance assumes the same covariance structure for all voxels. Here we relax this assumption for the first time by developing a version of PCA that allows the covariance structure to vary spatially. The new method is applied to real data and provides interesting new insight.

Wavelet footprints, proposed by Dragotti (2002), are used for speckle reduction of synthetic aperture radar (SAR) images. Wavelet footprints contain all wavelet coefficients associated with a singular structure of a signal. Consequently, the dependency across scales that is inherent in wavelet transformation is eliminated. In the present paper, coefficients of wavelet footprints are thresholded with hard thresholding. The denoising method shows great promise for speckle removal and hence provides good detection performance for SAR based recognition.

A linear feature extraction method based on the discrete wavelet transform (DWT) is applied. A binary genetic algorithm is used to select the best features from the different DWT representations in terms of cost. The feature extraction/selection methods are applied in classification of multisource remote sensing and geographic data. In experiments, the proposed methods performed well in terms of overall accuracies as compared to results obtained with other well-known feature extraction/selection methods.

Curvelet transform (CT), proposed by E. Candes et al. (1999), is used for speckle reduction of SAR images. The CT is useful for speckle reduction through its subband images and the speckle reduction is obtained by thresholding the subband-image coefficients of the digitized SAR images. Two thresholding methods are used; hard thresholding and soft thresholding. The denoising method shows great promise for speckle removal and hence provides good detection performance for SAR based recognition.