Hyperspectral image (HSI) super-resolution is commonly used to overcome the hardware limitations of existing hyperspectral imaging systems on spatial resolution. It fuses a low-resolution (LR) HSI and a high-resolution (HR) conventional image of the same scene to obtain an HR HSI. In this work, we propose a method that integrates a physical model and deep prior information. Specifically, a novel, yet effective two-stream fusion network is designed to serve as a regularizer for the fusion problem. This fusion problem is formulated as an optimization problem whose solution can be obtained by solving a Sylvester equation. Furthermore, the regularization parameter is simultaneously estimated to automatically adjust contribution of the physical model and the learned prior to reconstruct the final HR HSI. Experimental results on both simulated and real data demonstrate the superiority of the proposed method over other state-of-the-art methods on both quantitative and qualitative comparisons.

The first part of this paper presents some works conducted with Jose Bioucas Dias for fusing high spectral resolution images (such as hyperspectral images) and high spatial resolution images (such as panchromatic or multispectral images) in order to build images with improved spectral and spatial resolutions. These works are related to Bayesian fusion strategies exploiting prior information about the target image to be recovered constructed by dictionary learning. Interestingly, these Bayesian image fusion methods can be adapted with limited changes to motion estimation in pairs or sequences of images. The second part of this paper explains how the work of Jose Bioucas Dias has been a source of inspiration for developing new Bayesian motion estimation methods for ultrasound images.

Change detection (CD) is one of the most challenging issues when analyzing remotely sensed images. Comparing several multidate images acquired through the same kind of sensor is the most common scenario. Conversely, designing robust, flexible, and scalable algorithms for CD becomes even more challenging when the images have been acquired by two different kinds of sensors. This situation arises in the case of emergency under critical constraints. This paper presents, to the best of our knowledge, the first strategy to deal with optical images characterized by dissimilar spatial and spectral resolutions. Typical considered scenarios include CD between panchromatic, multispectral, and hyperspectral images. The proposed strategy consists of a three-step procedure: 1) inferring a high spatial and spectral resolution image by fusion of the two observed images characterized one by a low spatial resolution and the other by a low spectral resolution; 2) predicting two images with, respectively, the same spatial and spectral resolutions as the observed images by the degradation of the fused one; and 3) implementing a decision rule to each pair of observed and predicted images characterized by the same spatial and spectral resolutions to identify changes. To quantitatively assess the performance of the method, an experimental protocol is specifically designed, relying on synthetic yet physically plausible change rules applied to real images. The accuracy of the proposed framework is finally illustrated on real images.

This paper proposes a robust fusion-based strategy to detect changes between two multi-band optical images with different spatial and spectral resolutions, e.g., a multispectral high spatial resolution image and a hyperspectral low spatial resolution image. The dissimilarity between sensor resolutions makes the change detection problem challenging, which has been generally bypassed in the literature: most often, the two images are crudely and independently resampled in order to get the same spatial and spectral resolutions and finally, classical change detection methods are applied. However, the resampling operation tends to lose information. In this paper, we propose a method that more effectively uses the available information: the two observed images are respectively modeled as spatial and spectral degradations of two latent images characterized by the same high spatial and high spectral resolutions. Representing the same scene, these latent images are expected to be globally similar except for possible changes in sparse spatial locations. Change detection is then envisioned through the solution of an inverse problem, shown to be a specific instance of multi-band image fusion. The proposed method is applied to real images with simulated realistic changes. A comparison with state-of-the-art change detection methods evidences the proposed method superiority.

In the community of remote sensing, nonlinear mixture models have recently received particular attention in hyperspectral image processing. In this paper, we present a novel nonlinear spectral unmixing method following the recent multilinear mixing model of Heylen and Scheunders, which includes an infinite number of terms related to interactions between different endmembers. The proposed unmixing method is unsupervised in the sense that the endmembers are estimated jointly with the abundances and other parameters of interest, i.e., the transition probability of undergoing further interactions. Nonnegativity and sum-to-one constraints are imposed on abundances while only nonnegativity is considered for endmembers. The resulting unmixing problem is formulated as a constrained nonlinear optimization problem, which is solved by a block coordinate descent strategy, consisting of updating the end-members, abundances, and transition probability iteratively. The proposed method is evaluated and compared with existing linear and nonlinear unmixing methods for both synthetic and real hyperspectral data sets acquired by the airborne visible/infrared imaging spectrometer sensor. The advantage of using nonlinear unmixing as opposed to linear unmixing is clearly shown in these examples.

This letter proposes a robust fast multiband image fusion method to merge a high-spatial low-spectral resolution image and a low-spatial high-spectral resolution image. Following the method recently developed by Wei et al., the generalized Sylvester matrix equation associated with the multiband image fusion problem is solved in a more robust and efficient way by exploiting the Woodbury formula, avoiding any permutation operation in the frequency domain as well as the blurring kernel invertibility assumption required in their method. Thanks to this improvement, the proposed algorithm requires fewer computational operations and is also more robust with respect to the blurring kernel compared with the one developed by Wei et al. The proposed new algorithm is tested with different priors considered by Wei et al. Our conclusion is that the proposed fusion algorithm is more robust than the one by Wei et al. with a reduced computational cost.

This paper presents a multiband image fusion algorithm based on unsupervised spectral unmixing for combining a high-spatial-low-spectral-resolution image and a low-spatial-high-spectral-resolution image. The widely used linear observation model (with additive Gaussian noise) is combined with the linear spectral mixture model to form the likelihoods of the observations. The nonnegativity and sum-to-one constraints resulting from the intrinsic physical properties of the abundances are introduced as prior information to regularize this ill-posed problem. The joint fusion and unmixing problem is then formulated as maximizing the joint posterior distribution with respect to the endmember signatures and abundance maps. This optimization problem is attacked with an alternating optimization strategy. The two resulting subproblems are convex and are solved efficiently using the alternating direction method of multipliers. Experiments are conducted for both synthetic and semi-real data. Simulation results show that the proposed unmixing-based fusion scheme improves both the abundance and endmember estimation compared with the state-of-the-art joint fusion and unmixing algorithms.

This paper addresses the problem of single image super-resolution (SR), which consists of recovering a high-resolution image from its blurred, decimated, and noisy version. The existing algorithms for single image SR use different strategies to handle the decimation and blurring operators. In addition to the traditional first-order gradient methods, recent techniques investigate splitting-based methods dividing the SR problem into up-sampling and deconvolution steps that can be easily solved. Instead of following this splitting strategy, we propose to deal with the decimation and blurring operators simultaneously by taking advantage of their particular properties in the frequency domain, leading to a new fast SR approach. Specifically, an analytical solution is derived and implemented efficiently for the Gaussian prior or any other regularization that can be formulated into an l2 -regularized quadratic model, i.e., an l2 - l2 optimization problem. The flexibility of the proposed SR scheme is shown through the use of various priors/regularizations, ranging from generic image priors to learning-based approaches. In the case of non-Gaussian priors, we show how the analytical solution derived from the Gaussian case can be embedded into traditional splitting frameworks, allowing the computation cost of existing algorithms to be decreased significantly. Simulation results conducted on several images with different priors illustrate the effectiveness of our fast SR approach compared with existing techniques.

This paper presents a variational-based approach for fusing hyperspectral and multispectral images. The fusion problem is formulated as an inverse problem whose solution is the target image assumed to live in a lower dimensional subspace. A sparse regularization term is carefully designed, relying on a decomposition of the scene on a set of dictionaries. The dictionary atoms and the supports of the corresponding active coding coefficients are learned from the observed images. Then, conditionally on these dictionaries and supports, the fusion problem is solved via alternating optimization with respect to the target image (using the alternating direction method of multipliers) and the coding coefficients. Simulation results demonstrate the efficiency of the proposed algorithm when compared with state-of-the-art fusion methods.

This paper proposes a fast multi-band image fusion algorithm, which combines a high-spatial low-spectral resolution image and a low-spatial high-spectral resolution image. The well admitted forward model is explored to form the likelihoods of the observations. Maximizing the likelihoods leads to solving a Sylvester equation. By exploiting the properties of the circulant and downsampling matrices associated with the fusion problem, a closed-form solution for the corresponding Sylvester equation is obtained explicitly, getting rid of any iterative update step. Coupled with the alternating direction method of multipliers and the block coordinate descent method, the proposed algorithm can be easily generalized to incorporate prior information for the fusion problem, allowing a Bayesian estimator. Simulation results show that the proposed algorithm achieves the same performance as the existing algorithms with the advantage of significantly decreasing the computational complexity of these algorithms.

This paper presents a Bayesian fusion technique for multi-band images. The observed images are related to the high spectral and high spatial resolution image to be recovered through physical degradations, e.g., spatial and spectral blurring and/or subsampling defined by the sensor characteristics. The fusion problem is formulated within a Bayesian estimation framework. An appropriate prior distribution related to the linear mixing model for hyperspectral images is introduced. To compute Bayesian estimators of the scene of interest from its posterior distribution, a Gibbs sampling algorithm is proposed to generate samples asymptotically distributed according to the target distribution. To efficiently sample from this high-dimensional distribution, a Hamiltonian Monte Carlo step is introduced in this Gibbs sampler. The efficiency of the proposed fusion method is evaluated with respect to several state-of-the-art fusion techniques.

This paper studies a new Bayesian algorithm for fusing hyperspectral and multispectral images. The observed images are related to the high spatial resolution hyperspectral image to be recovered through physical degradations, e.g., spatial and spectral blurring and/or sub-sampling defined by the sensor characteristics. In this work, we assume that the spectral response of the multispectral sensor is unknown as it may not be available in practical applications. The resulting fusion problem is formulated within a Bayesian estimation framework, which is very convenient to model the uncertainty regarding the multispectral sensor characteristics and the scene to be estimated. The high spatial resolution hyperspectral image is then inferred from its posterior distribution. More precisely, to compute the Bayesian estimators associated with this posterior, a Markov chain Monte Carlo algorithm is proposed to generate samples asymptotically distributed according to the distribution of interest. Simulation results demonstrate the efficiency of the proposed fusion method when compared with several state-of-the-art fusion techniques.