In real hyperspectral scenes, heterogeneous spatial details and noises make a single subspace assumptions unrealistic. In this letter, a novel transferable multiple tensor subspace learning scheme is proposed for super-resolution enhancement of hyperspectral image (HSI). The intrinsic assumption is that the nonlocal patch tensors extracted from HSIs are derived from multiple tensor low-rank subspaces, which is compatible with practical data distribution and may better characterize the complex structures underlying HSIs. The transferable subspace structures are embedded into both nonblind and semi-blind HSI super-resolution. The alternating direction method of multipliers (ADMMs) algorithm is derived for model learning. The superiority of our method is demonstrated by comprehensive experiments on both synthetic and real datasets.

High-dimensional image representation is a challenging task since data has the intrinsic low-dimensional and shift-invariant characteristics. Currently, popular methods, such as tensor-Singular Value Decomposition (t-SVD), have limited ability in expressing shift-invariant subspace knowledge underlying data. To these problem, we propose a high-dimensional image representation framework based on Tensor Convolution-like Low-Rank Dictionary (TCLRD), which considers the shift-invariant low-dimensional structure of a tensor-valued data by convolution-like low-rank dictionary learning and coefficient coding, to promote the high-dimensional image representation ability. To be specific, we first define the TCLRD framework with low-rank constraint for dictionary and coefficient, in which tensor factorization and tensor-tensor product over frequency domain can be understood as convolution-like operation when describing shift-invariant. Then, the tensor Schatten-p norm is introduced to verify that TCLRD has rational mathematical interpretation. We study the TCLRD minimization problem in tensor completion with the ADMM-based optimization algorithm. The efficient solving scheme with TCLRD is extendable to various low-rank models like tensor robust principal component analysis and subspace clustering, and prove their theoretical guarantees based on generalization error. Extensive experimental results demonstrate the proposed TCLRD methods are beyond state-of-the-arts in typical tasks, including image denoising, HSI completion and image clustering.

The tensor nuclear norm (TNN), defined as the sum of nuclear norms of frontal slices of the tensor in a frequency domain, has been found useful in solving low-rank tensor recovery problems. Existing TNN-based methods use either fixed or data-independent transformations, which may not be the optimal choices for the given tensors. As the consequence, these methods cannot exploit the potential low-rank structure of tensor data adaptively. In this article, we propose a framework called self-adaptive learnable transform (SALT) to learn a transformation matrix from the given tensor. Specifically, SALT aims to learn a lossless transformation that induces a lower average-rank tensor, where the Schatten- p quasi-norm is used as the rank proxy. Then, because SALT is less sensitive to the orientation, we generalize SALT to other dimensions of tensor (SALTS), namely, learning three self-adaptive transformation matrices simultaneously from given tensor. SALTS is able to adaptively exploit the potential low-rank structures in all directions. We provide a unified optimization framework based on alternating direction multiplier method for SALTS model and theoretically prove the weak convergence property of the proposed algorithm. Experimental results in hyperspectral image (HSI), color video, magnetic resonance imaging (MRI), and COIL-20 datasets show that SALTS is much more accurate in tensor completion than existing methods. The demo code can be found at https://faculty.uestc.edu.cn/gaobin/zh_CN/lwcg/153392/list/index.htm.

Due to the influence of imaging equipment or environment, a hyperspectral image (HSI) is often unavoidably blurred in the acquisition process, which results in the spatial and spectral information loss of the HSI. The existing HSI deblurring methods can address the problem, however, they neglect the intrinsic structured sparsity and thus reduce the deblurring performance. Aiming at this issue, we propose a new HSI deblurring method based on transformed structured sparsity with smoothness (TSSS). We first use the local piecewise smoothness to obtain the spatial and spectral sparsity of an HSI in the gradient domain. Then, to capture the refined sparsity, we exploit the transform sparsity learning framework to encode the structured sparsity self-adaptively in transform space, where the sparse structures of transformed operators can be depicted by Laplacian scale mixture (LSM), i.e., the sparsity can be expressed as the product of a hidden positive scalar multiplier and a Laplacian vector. The visual and quantitative comparisons of experimental results on three HSI datasets indicate that our method outperforms state-of-the-arts.

Low-rank tensor completion has been widely used in computer vision and machine learning. This paper develops a novel multimodal core tensor factorization (MCTF) method combined with a tensor low-rankness measure and a better nonconvex relaxation form of this measure (NC-MCTF). The proposed models encode low-rank insights for general tensors provided by Tucker and T-SVD and thus are expected to simultaneously model spectral low-rankness in multiple orientations and accurately restore the data of intrinsic low-rank structure based on few observed entries. Furthermore, we study the MCTF and NC-MCTF regularization minimization problem and design an effective block successive upper-bound minimization (BSUM) algorithm to solve them. Theoretically, we prove that the iterates generated by the proposed models converge to the set of coordinatewise minimizers. This efficient solver can extend MCTF to various tasks such as tensor completion. A series of experiments including hyperspectral image (HSI), video and MRI completion confirm the superior performance of the proposed method.

Band selection (BS) reduces effectively the spectral dimension of a hyperspectral image (HSI) by selecting relatively few representative bands, which allows efficient processing in subsequent tasks. Existing unsupervised BS methods based on subspace clustering are built on matrix-based models, where each band is reshaped as a vector. They encode the correlation of data only in the spectral mode (dimension) and neglect strong correlations between different modes, i.e., spatial modes and spectral mode. Another issue is that the subspace representation of bands is performed in the raw data space, where the dimension is often excessively high, resulting in a less efficient and less robust performance. To address these issues, in this article, we propose a tensor-based subspace clustering model for hyperspectral BS. Our model is developed on the well-known Tucker decomposition. The three factor matrices and a core tensor in our model encode jointly the multimode correlations of HSI, avoiding effectively to destroy the tensor structure and information loss. In addition, we propose well-motivated heterogeneous regularizations (HRs) on the factor matrices by taking into account the important local and global properties of HSI along three dimensions, which facilitates the learning of the intrinsic cluster structure of bands in the low-dimensional subspaces. Instead of learning the correlations of bands in the original domain, a common way for the matrix-based models, our model learns naturally the band correlations in a low-dimensional latent feature space, which is derived by the projections of two factor matrices associated with spatial dimensions, leading to a computationally efficient model. More importantly, the latent feature space is learned in a unified framework. We also develop an efficient algorithm to solve the resulting model. Experimental results on benchmark datasets demonstrate that our model yields improved performance compared to the state-of-the-art.

Recently, tensor sparsity modeling has achieved great success in the tensor completion (TC) problem. In real applications, the sparsity of a tensor can be rationally measured by low-rank tensor decomposition. However, existing methods either suffer from limited modeling power in estimating accurate rank or have difficulty in depicting hierarchical structure underlying such data ensembles. To address these issues, we propose a parametric tensor sparsity measure model, which encodes the sparsity for a general tensor by Laplacian scale mixture (LSM) modeling based on three-layer transform (TLT) for factor subspace prior with Tucker decomposition. Specifically, the sparsity of a tensor is first transformed into factor subspace, and then factor sparsity in the gradient domain is used to express the local similarity in within-mode. To further refine the sparsity, we adopt LSM by the transform learning scheme to self-adaptively depict deeper layer structured sparsity, in which the transformed sparse matrices in the sense of a statistical model can be modeled as the product of a Laplacian vector and a hidden positive scalar multiplier. We call the method as parametric tensor sparsity delivered by LSM-TLT. By a progressive transformation operator, we formulate the LSM-TLT model and use it to address the TC problem, and then the alternating direction method of multipliers-based optimization algorithm is designed to solve the problem. The experimental results on RGB images, hyperspectral images (HSIs), and videos demonstrate the proposed method outperforms state of the arts.

Existing methods for tensor completion (TC) have limited ability for characterizing low-rank (LR) structures. To depict the complex hierarchical knowledge with implicit sparsity attributes hidden in a tensor, we propose a new multilayer sparsity-based tensor decomposition (MLSTD) for the low-rank tensor completion (LRTC). The method encodes the structured sparsity of a tensor by the multiple-layer representation. Specifically, we use the CANDECOMP/PARAFAC (CP) model to decompose a tensor into an ensemble of the sum of rank-1 tensors, and the number of rank-1 components is easily interpreted as the first-layer sparsity measure. Presumably, the factor matrices are smooth since local piecewise property exists in within-mode correlation. In subspace, the local smoothness can be regarded as the second-layer sparsity. To describe the refined structures of factor/subspace sparsity, we introduce a new sparsity insight of subspace smoothness: a self-adaptive low-rank matrix factorization (LRMF) scheme, called the third-layer sparsity. By the progressive description of the sparsity structure, we formulate an MLSTD model and embed it into the LRTC problem. Then, an effective alternating direction method of multipliers (ADMM) algorithm is designed for the MLSTD minimization problem. Various experiments in RGB images, hyperspectral images (HSIs), and videos substantiate that the proposed LRTC methods are superior to state-of-the-art methods.

We propose a novel graph Laplacian-guided coupled tensor decomposition (gLGCTD) model for fusion of hyperspectral image (HSI) and multispectral image (MSI) for spatial and spectral resolution enhancements. The coupled Tucker decomposition is employed to capture the global interdependencies across the different modes to fully exploit the intrinsic global spatial-spectral information. To preserve local characteristics, the complementary submanifold structures embedded in high-resolution (HR)-HSI are encoded by the graph Laplacian regularizations. The global spatial-spectral information captured by the coupled Tucker decomposition and the local submanifold structures are incorporated into a unified framework. The gLGCTD fusion framework is solved by a hybrid framework between the proximal alternating optimization (PAO) and the alternating direction method of multipliers (ADMM). Experimental results on both synthetic and real data sets demonstrate that the gLGCTD fusion method is superior to state-of-the-art fusion methods with a more accurate reconstruction of the HR-HSI.

Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named "structured sparse low-rank representation" (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation.

Conventional tensor completion (TC) methods generally assume that the sparsity of tensor-valued data lies in the global subspace. The so-called global sparsity prior is measured by the tensor nuclear norm. Such assumption is not reliable in recovering low-rank (LR) tensor data, especially when considerable elements of data are missing. To mitigate this weakness, this article presents an enhanced sparsity prior model for LRTC using both local and global sparsity information in a latent LR tensor. In specific, we adopt a doubly weighted strategy for nuclear norm along each mode to characterize global sparsity prior of tensor. Different from traditional tensor-based local sparsity description, the proposed factor gradient sparsity prior in the Tucker decomposition model describes the underlying subspace local smoothness in real-world tensor objects, which simultaneously characterizes local piecewise structure over all dimensions. Moreover, there is no need to minimize the rank of a tensor for the proposed local sparsity prior. Extensive experiments on synthetic data, real-world hyperspectral images, and face modeling data demonstrate that the proposed model outperforms state-of-the-art techniques in terms of prediction capability and efficiency.

Hyperspectral image (HSI) enjoys great advantages over more traditional image types for various applications due to the extra knowledge available. For the nonideal optical and electronic devices, HSI is always corrupted by various noises, such as Gaussian noise, deadlines, and stripings. The global correlation across spectrum (GCS) and nonlocal self-similarity (NSS) over space are two important characteristics for HSI. In this paper, a nonlocal low-rank regularized CANDECOMP/PARAFAC (CP) tensor decomposition (NLR-CPTD) is proposed to fully utilize these two intrinsic priors. To make the rank estimation more accurate, a new manner of rank determination for the NLR-CPTD model is proposed. The intrinsic GCS and NSS priors can be efficiently explored under the low-rank regularized CPTD to avoid tensor rank estimation bias for denoising performance. Then, the proposed HSI denoising model is performed on tensors formed by nonlocal similar patches within an HSI. The alternating direction method of multipliers-based optimization technique is designed to solve the minimum problem. Compared with state-of-the-art methods, the proposed algorithm can greatly promote the denoising performance of an HSI in various quality assessments.

Hyperspectral image (HSI) noise reduction is an active research topic in HSI processing due to its significance in improving the performance for object detection and classification. In this paper, we propose a joint spectral and spatial low-rank (LR) regularized method for HSI denoising, based on the assumption that the free-noise component in an observed signal can exist in latent low-dimensional structure while the noise component does not have this property. The proposed HSI denoising method not only considers the traditional LR property across the spectral domain but also leverages nonlocal LR property over the spatial domain. The main contribution of this paper is the incorporation of the low-rankness-based nonlocal similarity into sparse representation to characterize the spatial structure. Specially, the similar patches in each cluster usually contain similar sharp structure such as edges and textures; LR performed on cluster entitles to achieve a lower rank than that on the global spectral correlation. To make the proposed method more tractable and robust, we develop a variable splitting-based technique to solve the optimization problem. Experiment results on both simulated and real hyperspectral data sets demonstrate that the proposed method outperforms state-of-the-art methods with significant improvements both visually and quantitatively.

Sparsity-based methods have been widely used in hyperspectral imagery compression recovery (HSI-CR). However, most of the available HSI-CR methods work on vector space by vectorizing hyperspectral cubes in spatial and spectral domain, which will destroy spatial and spectral correlation and result in spatial and spectral information distortion in the recovery. At the same time, vectorization also make HSI's intrinsic structure sparsity cannot be utilized adequately. In this paper, a tensor non-local low-rank regularization (TNLR) approach is proposed to exploit essential structured sparsity and explore its advantages for CR of hyperspectral imagery. Specifically, a tensor nuclear norm penalty function is utilized as tensor low-rank regularization term to describe the spatial-and-spectral correlation hidden in HSI. To further improve the computational efficiency of the proposed algorithm, a fast implementation algorithm is developed by using the alternative direction multiplier method (ADMM) technique. Experimental results are shown that the proposed TNLR-CR algorithm can significantly outperform existing state-of-the-art CR techniques for hyperspectral image recovery.

This paper presents a technique that removes specular reflection in an image without sacrificing texture and chromaticity fidelity in diffuse regions of the image. This method is based on the observation that there exist structural similarity both in diffuse and specular components with original highlight image as well as strong local correlation in chromaticity in diffuse component. Local correlation in chromaticity regularizes chromaticity consistency. Structural similarity is measured using a gradient magnitude similarity map of a diffuse region. A diffuse component is estimated by solving local structure and chromaticity joint compensation problem. Experiment results show that the proposed method outperforms state-of-the-art specular reflection removal methods in terms of visual evaluation and the degree of information loss.