Recently, continuous transform-based tensor representation has emerged as a promising tool for multi-dimensional data recovery. However, the existing continuous transforms are essentially single-layer linear mappings, which limits their ability to capture the complex relationships inherent in multi-dimensional data. To overcome this limitation, we propose a Hierarchical Nonlinear Continuous Transform-based Tensor Representation (HiNCoT) for multi-dimensional data recovery. By leveraging the hierarchical nonlinear continuous transform, HiNCoT constructs the recovered tensor from a latent tensor, which is generated by the deep representation module with a low-rank core tensor as input. Compared with the existing continuous transform-based methods, HiNCoT can more effectively capture the complex nonlinear relationships inherent in multi-dimensional data along the third dimension. To evaluate the effectiveness of the proposed HiNCoT, we suggest an HiNCoT-based multi-dimensional data recovery model. Extensive experiments on diverse degeneration scenarios demonstrate the superiority of our hierarchical nonlinear transform-based method over existing single-layer linear transform-based methods.

Recently, tensor network (TN) decompositions have gained prominence in computer vision and contributed promising results to tensor recovery for their capability of compactly and efficiently representing high-order tensors. However, current TN topologies are rather being developed towards more intricate structures to pursue incremental improvements, resulting in a drastically increased number of TN ranks, which requires laborious hyper-parameter selection, especially for higher-order cases. In this paper, we propose a novel TN decomposition, dubbed tensor wheel (TW) decomposition, in which a high-order tensor is represented by a set of latent factors mapped into a specific wheel topology. Such a decomposition is constructed starting from analyzing the graph structure, aiming to more accurately characterize the complex interactions inside objectives while maintaining a lower hyper-parameter scale, theoretically alleviating the above deficiencies. The comprehensive analysis of the mathematical properties fully demonstrates that TW decomposition can be more potential in representation capabilities and more flexible in controlling both parameter storage and computational costs. To compute the TW-format decomposition, the sequential singular value decomposition (SVD)-based and the alternating least squares (ALS)-based learning algorithms are developed. Furthermore, to investigate the validity of TW decomposition, we provide its one numerical application, i.e., tensor completion (TC), yet develop an efficient proximal alternating minimization-based solving algorithm with guaranteed convergence. Experimental results on both synthetic and real-world data reveal that TW decomposition significantly outperforms other state-of-the-art tensor decompositions for incomplete-tensor inference, especially under solely few observations, thus substantiating the superiority and reliability of TW decomposition.

Spatio-temporal traffic data imputation is a fundamental component in intelligent transportation systems, which can significantly improve data quality and enhance the accuracy of downstream data mining tasks. Recently, low-rank tensor representation has shown great potential for spatio-temporal traffic data imputation. However, the low-rank assumption focuses on the global structure, neglecting the critical spatial topology and local temporal dependencies inherent in spatio-temporal data. To address these issues, we propose a topology-induced low-rank tensor representation (TILR), which can accurately capture the underlying low-rankness of the spatial multi-scale features induced by topology knowledge. Moreover, to exploit local temporal dependencies, we suggest a learnable convolutional regularization framework, which not only includes some classical convolution-based regularizers but also leads to the discovery of new convolutional regularizers. Equipped with the suggested TILR and convolutional regularizer, we build a unified low-rank tensor model harmonizing spatial topology and temporal dependencies for traffic data imputation, which is expected to deliver promising performance even under extreme and complex missing scenarios. To solve the proposed nonconvex model, we develop an efficient alternating direction method of multipliers (ADMM)-based algorithm and analyze its computational complexity. Extensive experiments demonstrate that the proposed model outperforms state-of-the-art baselines for various missing scenarios. These results reveal the critical synergy between topology-aware low-rank constraint and temporal dynamic modeling for spatio-temporal data imputation.

Hyperspectral pansharpening refers to fusing a panchromatic image (PAN) and a low-resolution hyperspectral image (LR-HSI) to obtain a high-resolution hyperspectral image (HR-HSI). Recently, guiding pre-trained diffusion models (DMs) has demonstrated significant potential in this area, leveraging their powerful representational abilities while avoiding complex training processes. However, these DMs are often trained on RGB images, not well-suited for pansharpening tasks, limited in adapting to the hyperspectral images. In this work, we propose a novel guided diffusion scheme with zero-shot guidance and neural spatialspectral decomposition (NSSD) to iteratively generate the RGB detail image and map the RGB detail image to target HR-HSI. Specifically, zero-shot guidance employs an auxiliary neural network that trained only with a PAN and LR-HSI to guide pre-trained DMs in generating the RGB detail image, informed by specific prior knowledge. Then, NSSD establishes a spectral mapping from the generated RGB detail image to the final HR-HSI. Extensive experiments are conducted on Pavia, Washington DC, Chukusei, and FR1 datasets to demonstrate that the proposed method significantly enhances the performance of DMs for hyperspectral pansharpening tasks, outperforming existing methods across multiple metrics and achieving improvements in visualization results. The code is available at https://github.com/Jin-liangXiao/DM-zs.

Medical decision-making often involves integrating knowledge from multiple clinical specialties, typically achieved through multidisciplinary teams.Inspired by this collaborative process, recent work has leveraged large language models (LLMs) in multi-agent collaboration frameworks to emulate expert teamwork.While these approaches improve reasoning through agent interaction, they are limited by static, pre-assigned roles, which hinder adaptability and dynamic knowledge integration.To address these limitations, we propose KAMAC, a Knowledge-driven Adaptive Multi-Agent Collaboration framework that enables LLM agents to dynamically form and expand expert teams based on the evolving diagnostic context.KAMAC begins with one or more expert agents and then conducts a knowledge-driven discussion to identify and fill knowledge gaps by recruiting additional specialists as needed.This supports flexible, scalable collaboration in complex clinical scenarios, with decisions finalized through reviewing updated agent comments.Experiments on two real-world medical benchmarks demonstrate that KAMAC significantly outperforms both single-agent and advanced multi-agent methods, particularly in complex clinical scenarios (i.e., cancer prognosis) requiring dynamic, cross-specialty expertise.

Multi-source image fusion combines the information coming from multiple images into one data, thus improving imaging quality. This topic has aroused great interest in the community. How to integrate information from different sources is still a big challenge, although the existing self-attention based transformer methods can capture spatial and channel similarities. In this paper, we first discuss the mathematical concepts behind the proposed generalized self-attention mechanism, where the existing self-attentions are considered basic forms. The proposed mechanism employs multilinear algebra to drive the development of a novel fully-connected self-attention (FCSA) method to fully exploit local and non-local domain-specific correlations among multi-source images. Moreover, we propose a multi-source image representation embedding it into the FCSA framework as a non-local prior within an optimization problem. Some different fusion problems are unfolded into the proposed fully-connected transformer fusion network (FC-Former). More specifically, the concept of generalized self-attention can promote the potential development of self-attention. Hence, the FC-Former can be viewed as a network model unifying different fusion tasks. Compared with state-of-the-art methods, the proposed FC-Former method exhibits robust and superior performance, showing its capability of faithfully preserving information.

Hyperspectral anomaly detection (HAD) aims to distinguish anomalies from background in hyperspectral images (HSIs). Recently, low-rank representation (LRR)-based methods have attracted widespread attention, but their pixel-level design overlooks semantic information in HSIs, resulting in unsatisfactory anomaly detection performance. To address this issue, we propose a prompted tensor decomposition (ProTD) for HAD, which cleverly leverages the power of large vision models with the designed prompt to guide the decomposition of HSIs. Specifically, guided by high-level semantic information distilled from a pretrained Segment Anything Model (SAM) with the prompt, we leverage uni-deep Tucker decomposition with Spatio-Spectral Total Variation (SSTV) regularization to represent the background, while using ℓ1norm to capture anomalies. To solve this model, we design an efficient algorithm based on alternating minimization. Extensive experiments on benchmark datasets (including Airport, Urban, and Beach) demonstrate that the proposed ProTD model outperforms state-of-the-art HAD methods, especially those LRR-based methods.

Thick cloud removal is a long-standing and critical challenge in remote sensing (RS) image processing, with the increasing availability of multi-temporal RS images offering new opportunities to address this problem. The main limitation of existing cloud removal methods is that the classical fidelity only considers original pixel domain or handcrafted/pretrained filtered domains, overlooking the individuality filters and the corresponding feature behind each RS image, which leads to evident detail discrepancies. To address this issue, we suggest a data-aware fidelity based on the untrained neural network, which encourages deep data-aware feature matching between the contaminated image and the guidance image. Complementary to the data-aware fidelity, we design the deep self-representation to implicitly impose regularization benefiting from the same untrained neural network. Equipped with the elaborately designed fidelity and regularization, we propose a unified data-aware fidelity and regularization learning (called DAFRL) paradigm for thick cloud removal that flexibly adapts to diverse multi-temporal RS images. Under this paradigm, the fidelity and regularization are empowered by the same untrained neural network, serving distinct functions while collaborating organically. Experimental results on both simulated and real datasets show that the proposed DAFRL effectively preserves fine details and outperforms the compared methods.

Tensor decompositions have been successfully applied to multidimensional data recovery. However, classical tensor decompositions are not suitable for emerging spatio-irregular multidimensional data (i.e., spatio-irregular tensor), whose spatial domain is non-rectangular, e.g., spatial transcriptomics data from bioinformatics and semantic units from computer vision. By using preprocessing (e.g., zero-padding or element-wise 0-1 weighting), the spatio-irregular tensor can be converted to a spatio-regular tensor and then classical tensor decompositions can be applied, but this strategy inevitably introduces bias information, leading to artifacts. How to design a tensor-based method suitable for emerging spatio-irregular tensors is an imperative challenge. To address this challenge, we propose a learnable transform-assisted tensor singular value decomposition (LTA-TSVD) for spatio-irregular tensor recovery, which allows us to leverage the intrinsic structure behind the spatio-irregular tensor. Specifically, we design a learnable transform to project the original spatio-irregular tensor into its latent spatio-regular tensor, and then the latent low-rank structure is captured by classical TSVD on the resulting regular tensor. Empowered by LTA-TSVD, we develop spatio-irregular low-rank tensor completion (SIR-LRTC) and spatio-irregular tensor robust principal component analysis (SIR-TRPCA) models for the spatio-irregular tensor imputation and denoising respectively, and we design corresponding solving algorithms with theoretical convergence. Extensive experiments including the spatial transcriptomics data imputation and hyperspectral image denoising show SIR-LRTC and SIR-TRPCA are superior performance to competing approaches and benefit downstream applications.

Recently, the tensor nuclear norm (TNN)-based tensor robust principle component analysis (TRPCA) has achieved impressive performance in multidimensional data processing. The underlying assumption in TNN is the low-rankness of frontal slices of the tensor in the transformed domain (e.g., Fourier domain). However, the low-rankness assumption is usually violative for real-world multidimensional data (e.g., video and image) due to their intrinsically nonlinear structure. How to effectively and efficiently exploit the intrinsic structure of multidimensional data remains a challenge. In this article, we first suggest a kernelized TNN (KTNN) by leveraging the nonlinear kernel mapping in the transform domain, which faithfully captures the intrinsic structure (i.e., implicit low-rankness) of multidimensional data and is computed at a lower cost by introducing kernel trick. Armed with KTNN, we propose a tensor robust kernel PCA (TRKPCA) model for handling multidimensional data, which decomposes the observed tensor into an implicit low-rank component and a sparse component. To tackle the nonlinear and nonconvex model, we develop an efficient alternating direction method of multipliers (ADMM)-based algorithm. Extensive experiments on real-world applications collectively verify that TRKPCA achieves superiority over the state-of-the-art RPCA methods.

Tensor network (TN) representation is a powerful technique for computer vision and machine learning. TN structure search (TN-SS) aims to search for a customized structure to achieve a compact representation, which is a chal-lenging NP-hard problem. Recent “sampling-evaluation”-based methods require sampling an extensive collection of structures and evaluating them one by one, resulting in pro-hibitively high computational costs. To address this issue, we propose a novel TN paradigm, named SVD-inspired TN decomposition (SVDinsTN), which allows us to efficiently solve the TN-SS problem from a regularized modeling per-spective, eliminating the repeated structure evaluations. To be specific, by inserting a diagonal factor for each edge of the fully-connected TN, SVDinsTN allows us to calculate TN cores and diagonal factors simultaneously, with the factor sparsity revealing a compact TN structure. In theory, we prove a convergence guarantee for the proposed method. Experimental results demonstrate that the proposed method achieves approximately 100 ~ 1000 times acceleration compared to the state-of-the-art TN-SS methods while maintaining a comparable level of representation ability.

Recently, subspace representation-based methods have shown notable success in hyperspectral image (HSI) denoising by exploiting the spectral correlation of HSIs. However, these methods cannot fully explore the spatial correlation within HSIs, resulting in unsatisfactory results. To address this limitation, we propose a novel generalized Kronecker decomposition-based subspace representation (GKDSR) method, which can simultaneously characterize the spatial and spectral correlation of HSIs. Specifically, we first decompose an HSI as the spectral subspace and coefficients. Then we impose the face-wise generalized Kronecker decomposition to the coefficients to fully characterize the spatial correlation of the HSI. Armed with the proposed GKDSR method, we build a GKDSR-based HSI denoising model, which can not only achieve a promising spectral fidelity but also preserve spatial fine details. To solve the proposed HSI denoising model, we develop an efficient alternating minimization-based algorithm. Experimental results demonstrate the significant superiority of the proposed method compared with competing methods in terms of spectral fidelity and preservation of spatial details.

Pansharpening aims to obtain the high resolution multispectral image (HRMS) using the panchromatic image (PAN) and low spatial resolution multispectral image (LRMS). The similarity between PAN and HRMS has shown powerful performance for spatial feature extraction. The prevailing methods usually describe the similarity on a fixed transformed domain. However, such domain, e.g., gradient domain, usually limits the preservation of spatial details and neglects flexibility. To overcome these challenges, we propose an adaptive transformed domain-based spatial fidelity to depict the similarity accurately and flexibly. Based on the proposed spatial fidelity, we build a novel variational pansharpening model that consists of spectral and spatial fidelity terms. We design an algorithm based on the alternating direction method of multiplier (ADMM) framework to solve the model. Experimental results on reduced- and full-resolution data verify the effectiveness of the proposed method.

Tensor decomposition-based models have received increasing attention in hyperspectral image (HSI) denoising. However, tensor decompositions (e.g., Tucker decomposition and tensor singular value decomposition) in these HSI denoising models ignore exploiting the multiple components of the HSI, resulting in unsatisfactory denoising performance. To fully exploit the multiple components of the HSI, we develop a sparsity regularized rank-(L,M,N) block term decomposition (SR-BTD). In SR-BTD, the clean HSI is decomposed as the sum of multiple components, where each component is a sparse core tensor multiplied by matrices along each mode. The sparse regularization on each core tensor can benefit determining the low-rankness of each component with the unknown rank-(L,M,N) in the real world, leading to more accurately exploiting each component. Equipped with SR-BTD, we establish the HSI denoising model and design a hierarchical alternating least squares-based algorithm to efficiently solve the resulting model. Extensive experiments on simulated and real HSI denoising tasks demonstrate SR-BTD is superior to the competing tensor decompositions in terms of numerical results and visual quality.

The pixel intensity of remote sensing images at different time nodes exhibits significant differences due to factors such as changes in solar illumination angles. Consequently, the previous cloud removal methods, primarily based on the original pixel domain, yield unsatisfactory results. In this paper, considering the sharing of similar features among remote sensing images at different time nodes, we first design a novel feature-domain fidelity that leverages the feature extraction capability of convolution operator, allowing for the precise preservation of intricate details and textures inherent in multi-temporal remote sensing images. Building upon the feature-domain fidelity, we propose a cloud removal model that organically integrates the low fully-connected tensor network rank regularization, which comprehensively captures the spatial-spectral-temporal correlations of multi-temporal remote sensing images. Moreover, we develop an effective algorithm based on proximal alternating minimization to solve the proposed model. Numerical experiments conducted on both simulated and real-world data validate that the proposed method outperforms the compared ones.

Many real-world images (e.g., hyperspectral images (HSIs) and color videos) are usually partially observed and contaminated by Poisson noise, which hinder their subsequent applications. Recently, the tensor singular value decomposition (t-SVD)-based model was suggested for image recovery with Poisson observation. However, the classic t-SVD usually fails to capture the complex nonlinear structure of real-world images. To address this problem, we suggest an attention-guided low-rank tensor factorization (AGLRTF)-based model for image recovery with Poisson observation. More concretely, we consider a self-attention network as the transform in the t-SVD framework, which can treat the frontal slices unequally, allowing us to enhance the low rankness of the transformed frontal slices. Also, the self-attention block is learned unsupervised from the given data. Extensive experiments on HSIs demonstrate that our method achieves approximately a 2-dB higher PSNR metric compared with state-of-the-art methods.

Recently, deep learning has been widely applied in the field of blind hyperspectral unmixing (HU), which aims to simultaneously estimate constitutive endmembers and their abundances in hyperspectral images (HSIs). Generally, the HU process based on deep-learning methods consists of two parts: an encoder and a decoder. In many networks, the decoder stage uses the extracted semantic information of the HSI by the encoder, without direct access to the manifold structure of the HSI. To address this limitation and simultaneously capture both the semantic information and manifold structure of the HSI, in this letter, we propose a dual-channel enhanced decoder network (DED-Net) for the HU problem. Specifically, DED-Net redesigns a decoder by adding a dual-channel graph regularizer that establishes a physically meaningful immediate connection between the abundance and the HSI, effectively integrating both the information from the encoder and the original HSI to enhance endmembers and abundance estimation. Experimental results demonstrate the superiority of our proposed method, which leads to a more accurate unmixing performance.

Multi-dimensional data are inevitably corrupted, which hinders subsequent applications (e.g., image segmentation and classification). Recently, due to the powerful ability to characterize the correlation between any two modes of tensors, fully-connected tensor network (FCTN) decomposition has received increasing attention in multi-dimensional data recovery. However, the expressive power of FCTN decomposition in the original pixel domain has yet to be fully leveraged, which can not provide satisfactory results in the recovery of details and textures, especially for low-sampling rates or heavy noise scenarios. In this work, we suggest a feature-based FCTN decomposition model (termed as F-FCTN) for multi-dimensional data recovery, which can faithfully capture the relationship between the spatial-temporal/spectral-feature modes. Compared with the original FCTN decomposition, F-FCTN can more effectively recover the details and textures and be more suitable for the subsequent high-level applications. However, F-FCTN leads to a larger-scale feature tensor as compared with the original tensor, which brings challenges in designing the solving algorithm. To harness the resulting large-scale optimization problem, we develop an efficient leverage score sampling-based proximal alternating minimization (S-PAM) algorithm and theoretically establish its relative error guarantee. Extensive numerical experiments on real-world data illustrate that the proposed method performs favorably against compared methods in data recovery and facilitates subsequent image classification.

Recently, fully-connected tensor network (FCTN) decomposition, which factorizes the target tensor into a series of interconnected factor tensors, has drawn growing focus on multi-dimensional visual data processing. However, the lack of clear physical interpretation for the factor tensors hinders us from introducing handcrafted regularizers to deeply explore the potential of FCTN decomposition. To tackle this issue, we suggest a unimode hierarchical nonlinear (UHN) decomposition for each factor tensor, which can adaptively capture the complex nonlinear structure and implicitly regularize factor tensors. With this UHN decomposition of the factor tensors, we naturally propose a nested fully-connected tensor network (N-FCTN) decomposition. Attributed to the adaptive and implicit regularization inherent in UHN decomposition of factor tensors, the proposed N-FCTN decomposition is expected to perform favorably against the original FCTN decomposition. Based on the proposed N-FCTN decomposition, we build a multi-dimensional visual data recovery model and provide a theoretical error bound between the recovered tensor by our model and the underlying tensor. To address the resulting non-convex and nonlinear optimization problem, we develop an efficient proximal alternating minimization (PAM)-based algorithm and establish its theoretical convergence guarantee. Extensive experimental results on multi-spectral images, color videos, and light field data demonstrate the superior recovery performance of the proposed method compared to the state-of-the-art methods.

Thick cloud contamination in remote sensing (RS) images significantly hinders their utility in subsequent applications. Traditional cloud removal methods predominantly focus on the design of the regularization term, while neglecting the design of the fidelity term. Recently, the proposal of gradient-domain fidelity has highlighted the significance of the fidelity term design, dedicated to maintaining textures in the gradient domain. However, the handcrafted gradient-domain fidelity still has limitations in capturing invariant and delicate features behind multitemporal RS (MTRS) images, leading to unsatisfactory detail preservation. To address the problem, we suggest a deep domain fidelity to capture deep features by leveraging a pretrained deep network, which matches the intrinsic deep feature between the original images and reference images, instead of matching the shallow features in gradient domain fidelity. Empowered with the deep domain fidelity, we propose a thick cloud removal model (called DFTR) for MTRS images, organically integrating the deep domain fidelity term with a low-rank (LR) regularization term (i.e., tensor ring (TR) decomposition), offering fine detail preservation. Extensive simulated and real experiments on MTRS images demonstrate that the proposed method outperforms the compared methods, including the origin domain-based and the gradient domain-based methods, in thick cloud removal, especially for detail preservation.

Hyperspectral image super-resolution (HSI SR) aims to combine the detailed spectral information of hyperspectral images with the spatial resolution of multispectral images, thus enhancing the ability to extract valuable insights across various applications. Recently, the tensor singular value decomposition (t-SVD) has emerged as a powerful tool and has been introduced into the HSI SR field for exploring low-rank prior information. For t-SVD, the domain transform is crucial to acquiring more low-rank data characteristics. Nevertheless, previous efforts on domain transform have only involved the single transformed domain (i.e., single domain), while ignoring the potential pursuing the lower rankness in multiple successional transformed domains, termed cross-domain (CD). In this article, we propose a novel CD-based t-SVD and define the corresponding tensor CD rank based on a pivotal observation, i.e., the low-rank behavior of HSI in CD is more significant than that in single domain. More specifically, we first define a successional linear transform (SLT) to establish the CD concept, then develop a novel CD-based t-SVD and tensor CD rank, and theoretically deduce a new tensor CD-nuclear norm as the convex approximation of CD rank. Equipped with such a CD rank, we thus formulate a CD-rank-constrained minimization model for the HSI SR task, called CroDoSR, which is effectively solved by the alternating direction method of multipliers (ADMMs). Comprehensive experiments on several widely used datasets evidently demonstrate the superiority of the proposed CroDoSR method.

Hyperspectral and multispectral image fusion, denoted as HSI-MSI fusion, involves merging a pair of hyperspectral (HSI) and multispectral (MSI) images to generate a high spatial resolution hyperspectral image (HR-HSI). The primary challenge in HSI-MSI fusion is to find the best way to extract one-dimensional spectral features and two-dimensional (2-D) spatial features from HSI and MSI and harmoniously combine them. In recent times, coupled tensor decomposition (CTD)-based methods have shown promising performance in the fusion task. However, the tensor decompositions (TDs) used by these CTD-based methods face difficulties in extracting complex features and capturing 2-D spatial features, resulting in suboptimal fusion results. To address these issues, we introduce a novel method called Coupled Tensor Double-Factor Decomposition (CTDF). Specifically, we propose a Tensor Double-Factor (TDF) decomposition, representing a 3rd-order HR-HSI as a 4th-order spatial factor and a 3rd-order spectral factor, connected through tensor contraction. Compared to other TDs, the TDF has better feature extraction capability since it has a higher order factor than that of HR-HSI, whereas the other TDs only have the same order factor as the HR-HSI. Moreover, the TDF can extract 2-D spatial features using the 4th-order spatial factor. We apply the TDF to the HSI-MSI fusion problem and formulate the CTDF model. Furthermore, we design an algorithm based on proximal alternating minimization to solve this model and provide insights into its computational complexity and convergence analysis. The simulated and real experiments validate the effectiveness and efficiency of the proposed CTDF method. The code is available at https://github.com/tingxu113/CTDF.

Recently, the transform-based tensor nuclear norm (TNN) methods have shown promising performance and drawn increasing attention in tensor completion (TC) problems. The main idea of these methods is to exploit the low-rank structure of frontal slices of the tensor under the transform. However, the transforms in TNN methods usually treat all modes equally and do not consider the different traits of different modes (i.e., spatial and spectral/temporal modes). To address this problem, we suggest a new low-rank tensor representation based on the coupled nonlinear transform (called CoNoT) for a better low-rank approximation. Concretely, spatial and spectral/temporal transforms in the CoNoT, respectively, exploit the different traits of different modes and are coupled together to boost the implicit low-rank structure. Here, we use the convolutional neural network (CNN) as the CoNoT, which can be learned solely from an observed multidimensional image in an unsupervised manner. Based on this low-rank tensor representation, we build a new multidimensional image completion model. Moreover, we also propose an enhanced version (called Ms-CoNoT) to further exploit the spatial multiscale nature of real-world data. Extensive experiments on real-world data substantiate the superiority of the proposed models against many state-of-the-art methods both qualitatively and quantitatively.

Recently, untrained neural network priors (UNNPs) have received increasing attention for multi-dimensional image recovery. However, previous studies are based on over-parameterized untrained neural networks, which results in unstable behavior. In this letter, we propose an untrained low-rank neural network prior (ULRNNP) for multi-dimensional image recovery, which enjoys the powerful representation ability and stable behavior. More specifically, the elaborately designed nonlinear Tucker decomposition module implicitly imposes low-rank constraints on the feature tensor and can more compactly represent the feature tensor. Attributed to the suggested nonlinear Tucker decomposition module, ULRNNP can simultaneously enjoy strong representation ability and stable behavior. The friendly stable behavior allows us to design a friendly stopping criteria without the reference ground truth image as compared with classic UNNP-based methods. Extensive experiments on different multi-dimensional image datasets validate the superior performance of the proposed ULRNNP over state-of-the-art methods.

Thick clouds in remote sensing (RS) images deteriorate the visual quality and hinder subsequent applications. The emerging multitemporal RS images with rich temporal information bring the opportunity for cloud removal. How to effectively exploit the rich temporal information of the multitemporal RS images remains challenging. As multitemporal RS images with the same geographic scene, the spatial gradient of RS images at different time nodes has a resemblance, which can guide the reconstruction of the cloudy region. Motivated by this, we suggest a gradient domain fidelity with respect to the guided gradient for thick cloud removal in multitemporal RS images, which faithfully preserves the fine edges and textures compared to the original pixel domain fidelity. Armed with the gradient domain fidelity, we propose a low-rank tensor ring decomposition model (TRGFid) for the thick cloud removal problem. In the proposed model, the guided gradient of the cloudy region is availably estimated by using the regression method from the cloud-free region of different time nodes. Moreover, we develop an efficient proximal alternating minimization-based algorithm for solving the proposed nonconvex model. Extensive simulated and real experiments show that the proposed method outperforms its competitors, and preserves fine edges and textures.

Cloud removal is an important task in the remotely sensed images (RSIs) processing, which is beneficial for downstream applications, such as unmixing, fusion, and target detection. Multi-temporal remotely sensed images (MRSIs), which contains the abundant spatial-spectral-temporal (SST) information, potentially bring the new opportunities for cloud removal. However, how to effectively and efficiently explore the rich information of MRSIs remains a challenge. Inspired by the low-rankness of MRSIs, we propose an Unsupervised Domain Factorization Network (UnDFN) for thick cloud removal, which allows us to effectively and efficiently exploit the rich SST information of MRSIs. In UnDFN framework, we first factorize RSI for each time node of MRSIs into its corresponding spatial factor and spectral factor. Due to the powerful expressive ability, the untrained neural networks are leveraged to faithfully capture the spatial and spectral factors. Especially, motivated by the low-rankness of the concatenated spatial factors of all time nodes, a low-rank spatial factor module is elaborately designed to effectively and efficiently capture the spatial factors of all time nodes as compared with separately using networks to capture spatial factors for each time node. Extensive experiments on simulated and real MRSIs of different satellites (including Sentinel-2 and Landsat-8) substantiate that the proposed UnDFN achieves state-of-the-art performance in thick cloud removal compared to other methods.

Pansharpening (which stands for panchromatic sharpening) involves the fusion between a multispectral (MS) image with a higher spectral content than a fine spatial resolution panchromatic (PAN) image to generate a high spatial resolution multispectral (HRMS) image. A widely-used concept is the construction of the relationship between PAN and HRMS images by designing pixel-based coefficients. Previous pixel-based methods compute the coefficients pixel-by-pixel while suffering from inaccuracies in some areas leading to spatial distortion. However, we found that the coefficients inherit the spatial properties of the HRMS image, e.g., the local smoothness and nonlocal self-similarity, and the spatial correlation between the coefficients and the HRMS image can increase the accuracy of the estimation process. In this article, we propose a novel spatial fidelity with nonlocal regression (SFNLR) to describe the relationship between PAN and HRMS images. Unlike from the pixel-based perspective, the SFNLR can jointly utilize the local smoothness and nonlocal self-similarity of the coefficients for preserving spatial information. Besides, the SFNLR is integrated with a widely-used spectral fidelity to formulate a new variational model for the pansharpening problem. An effective algorithm based on the alternating direction method of multiplier (ADMM) framework is designed to solve the proposed model. Qualitative and quantitative assessments on reduced and full resolution datasets from different satellites demonstrate that the proposed approach outperforms several state-of-the-art methods. The code is available at: https://github.com/Jin-liangXiao/SFNLR.

Recently, the transform-based tensor nuclear norm (TNN) framework has yielded promising results for hyperspectral image (HSI) denoising as compared with previous original-domain tensor-based models. However, the TNN framework only exploits the low-rankness of each band of HSIs (tensors) under a single spectral transform. The correlation between all bands under the transform (i.e., the global low-rankness of the transformed tensor) and the sparsity of the transformed HSI, which are beneficial for HSI denoising, is usually neglected in the TNN framework. In this article, we propose to reconcile sparse and low-tensor-ring (TR)-rank priors in the learned transformed domain (called T-RSTR model) for HSI denoising. In T-RSTR, the transform-based low-TR-rank and sparse regularizers are designed to characterize the global low-rankness and sparsity of the transformed tensors, respectively, and then the transform-based low-TR-rank and sparse regularizers are organically integrated and benefit from each other for substantially boosting denoising performance. To tackle the T-RSTR model, we elaborately design a proximal alternating minimization-based algorithm with the theoretical convergence. Extensive numerical results demonstrate that T-RSTR is superior to the competing methods.

Pansharpening refers to the fusion of a low spatial-resolution multispectral image with a high spatial-resolution panchromatic image. In this paper, we propose a novel low-rank tensor completion (LRTC)-based framework with some regularizers for multispectral image pansharpening, called LRTCFPan. The tensor completion technique is commonly used for image recovery, but it cannot directly perform the pansharpening or, more generally, the super-resolution problem because of the formulation gap. Different from previous variational methods, we first formulate a pioneering image super-resolution (ISR) degradation model, which equivalently removes the downsampling operator and transforms the tensor completion framework. Under such a framework, the original pansharpening problem is realized by the LRTC-based technique with some deblurring regularizers. From the perspective of regularizer, we further explore a local-similarity-based dynamic detail mapping (DDM) term to more accurately capture the spatial content of the panchromatic image. Moreover, the low-tubal-rank property of multispectral images is investigated, and the low-tubal-rank prior is introduced for better completion and global characterization. To solve the proposed LRTCFPan model, we develop an alternating direction method of multipliers (ADMM)-based algorithm. Comprehensive experiments at reduced-resolution (i.e., simulated) and full-resolution (i.e., real) data exhibit that the LRTCFPan method significantly outperforms other state-of-the-art pansharpening methods. The code is publicly available at: https://github.com/zhongchengwu/code_LRTCFPan.

Pansharpening refers to the fusion of a panchromatic (PAN) image with a high spatial resolution and a multispectral (MS) image with a low spatial resolution, aiming to obtain a high spatial resolution MS (HRMS) image. In this article, we propose a novel deep neural network architecture with level-domain-based loss function for pansharpening by taking into account the following double-type structures, i.e., double-level, double-branch, and double-direction, called as triple-double network (TDNet). By using the structure of TDNet, the spatial details of the PAN image can be fully exploited and utilized to progressively inject into the low spatial resolution MS (LRMS) image, thus yielding the high spatial resolution output. The specific network design is motivated by the physical formula of the traditional multi-resolution analysis (MRA) methods. Hence, an effective MRA fusion module is also integrated into the TDNet. Besides, we adopt a few ResNet blocks and some multi-scale convolution kernels to deepen and widen the network to effectively enhance the feature extraction and the robustness of the proposed TDNet. Extensive experiments on reduced- and full-resolution datasets acquired by WorldView-3, QuickBird, and GaoFen-2 sensors demonstrate the superiority of the proposed TDNet compared with some recent state-of-the-art pansharpening approaches. An ablation study has also corroborated the effectiveness of the proposed approach. The code is available at https://github.com/liangjiandeng/TDNet.

Despite the success of vision Transformers for the image deraining task, they are limited by computation-heavy and slow runtime. In this work, we investigate Transformer decoder is not necessary and has huge computational costs. Therefore, we revisit the standard vision Transformer as well as its successful variants and propose a novel Decoder-Free Transformer-Like (DFTL) architecture for fast and accurate single image deraining. Specifically, we adopt a cheap linear projection to represent visual information with lower computational costs than previous linear projections. Then we replace standard Transformer decoder block with designed Progressive Patch Merging (PPM), which attains comparable performance and efficiency. DFTL could significantly alleviate the computation and GPU memory requirements through proposed modules. Extensive experiments demonstrate the superiority of DFTL compared with competitive Transformer architectures, e.g., ViT, DETR, IPT, Uformer, and Restormer. The code is available at https://github.com/XiaoXiao-Woo/derain.

Recently, tensor network (TN) decompositions have gained prominence in computer vision and contributed promising results to high-order data recovery tasks. However, current TN models are rather being developed towards more intricate structures to pursue incremental improvements, which instead leads to a dramatic increase in rank numbers, thus encountering laborious hyper-parameter selection, especially for higher-order cases. In this paper, we propose a novel TN decomposition, dubbed tensor wheel (TW) decomposition, in which a high-order tensor is represented by a set of latent factors mapped into a specific wheel topology. Such decomposition is constructed starting from analyzing the graph structure, aiming to more accurately characterize the complex interactions inside objectives while maintaining a lower hyper-parameter scale, theoretically alleviating the above deficiencies. Furthermore, to investigate the potentiality of TW decomposition, we provide its one numerical application, i.e., tensor completion (TC), yet develop an efficient proximal alternating minimization-based solving algorithm with guaranteed convergence. Experimental results elaborate that the proposed method is significantly superior to other tensor decomposition-based state-of-the-art methods on synthetic and real-world data, implying the merits of TW decomposition. The code is available at: https://github.com/zhongchengwu/code_TWDec.

Hyperspectral image super-resolution (HISR) is to fuse a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI), aiming to obtain a high-resolution hyperspectral image (HR-HSI). Recently, various convolution neural network (CNN) based techniques have been successfully applied to address the HISR problem. However, these methods generally only consider the relation of a local neighborhood by convolution kernels with a limited receptive field, thus ignoring the global relationship in a feature map. In this paper, we design a transformer-based architecture (called Fusformer) for the HISR problem, which is the first attempt to apply the transformer architecture to this task to the best of our knowledge. Thanks to the excellent ability of feature representations, especially by the self-attention in the transformer, our approach can globally explore the intrinsic relationship within features. Considering the specific HISR problem, since the LR-HSI holds the primary spectral information, our method estimates the spatial residual between the upsampled LR-MSI and the desired HR-HSI, reducing the burden of training the whole data in a smaller mapping space. Various experiments show that our approach outperforms current state-of-the-art HISR methods. The code is available at https://github.com/J-FHu/Fusformer.

Recently, low-rank prior (LRP) has achieved success in tensor completion, thanks to its ability in capturing global correlations. However, since the presence of dynamic foreground breaks the low-rank assumption, LRP sometimes fails to work for complex videos. How to effectively capture the implicit low-rankness is a tricky challenge in complex video completion. To address this challenge, we propose a video completion model, which organically combines the respective merits of deterministic LRP and deep image prior. Specifically, we first decompose a complex video into the dynamic foreground and static background, and then characterize each component individually. For the dynamic foreground, instead of the hand-crafted prior, we consider the deep foreground prior expressed by U-Net architecture in an unsupervised manner, while we consider the tensor low-rank prior for the static background. Moreover, we develop an efficient alternating direction method of multipliers algorithm under the plug-and-play framework to tackle the resulting model. Different from the classic plug-and-play framework with a pre-trained and fixed network, our algorithm can adaptively update the parameters of network during iterations, which enables the network to have a more flexible expression ability. Experimental results verify that the proposed method can do better than some state-of-the-art tensor completion methods in complex video completion.

The general tensor-based methods can recover missing values of multidimensional images by exploiting the low-rankness on the pixel level. However, especially when considerable pixels of an image are missing, the low-rankness is not reliable on the pixel level, resulting in some details losing in their results, which hinders the performance of subsequent image applications (e.g., image recognition and segmentation). In this article, we suggest a novel multiscale feature (MSF) tensorization by exploiting the MSFs of multidimensional images, which not only helps to recover the missing values on a higher level, that is, the feature level but also benefits subsequent image applications. By exploiting the low-rankness of the resulting MSF tensor constructed by the new tensorization, we propose the convex and nonconvex MSF tensor train rank minimization (MSF-TT) to conjointly recover the MSF tensor and the corresponding original tensor in a unified framework. We develop the alternating directional method of multipliers (ADMMs) to solve the convex MSF-TT and the proximal alternating minimization (PAM) to solve the nonconvex MSF-TT. Moreover, we establish the theoretical guarantee of convergence for the PAM algorithm. Numerical examples of real-world multidimensional images show that the proposed MSF-TT outperforms other compared approaches in image recovery and the recovered MSF tensor can benefit the subsequent image recognition.

Hyperspectral image (HSI) mixed noise removal is a fundamental problem and an important preprocessing step in remote sensing fields. The low-rank approximation-based methods have been verified effective to encode the global spectral correlation for HSI denoising. However, due to the large scale and complexity of real HSI, previous low-rank HSI denoising techniques encounter several problems, including coarse rank approximation (such as nuclear norm), the high computational cost of singular value decomposition (SVD) (such as Schatten$p$-norm), and adaptive rank selection (such as low-rank factorization). In this article, two novel factor group sparsity-regularized nonconvex low-rank approximation (FGSLR) methods are introduced for HSI denoising, which can simultaneously overcome the mentioned issues of previous works. The FGSLR methods capture the spectral correlation via low-rank factorization, meanwhile utilizing factor group sparsity regularization to further enhance the low-rank property. It is SVD-free and robust to rank selection. Moreover, FGSLR is equivalent to Schatten$p$-norm approximation (Theorem 1), and thus FGSLR is tighter than the nuclear norm in terms of rank approximation. To preserve the spatial information of HSI in the denoising process, the total variation regularization is also incorporated into the proposed FGSLR models. Specifically, the proximal alternating minimization is designed to solve the proposed FGSLR models. Experimental results have demonstrated that the proposed FGSLR methods significantly outperform existing low-rank approximation-based HSI denoising methods.

Hyperspectral image (HSI) denoising has been regarded as an effective and economical preprocessing step in data subsequent applications. Recent nonlocal low-rank approximation on each full band patch group has demonstrated their superiority for HSI denoising. These methods, however, directly design the low-rank regularization to the grouped patch image itself (i.e., original domain), which ignores the spatial information of the grouped patch image and cannot explores the potential structure. To address these issues, this paper proposes a nonlocal group sparsifying transform learning method (dubbed TLNLGS) for HSI denoising. Motivated by the global spectral correlation in the HSI, we firstly impose a certain low-dimensional subspace hypothesis over the HSI to prevent the heavy computation burden with the spectral band increases, and then explore a discriminatively intrinsic nonlocal group sparse prior of the reduced image by transform model. The learned group sparse prior can not only excavate the nonlocal self-similarity as recent nonlocal low-rank approximation methods but also preserve the local spatial smooth structure of the image. Moreover, compared with the fixed transform domain (e.g., gradient and discrete cosine transformation domains), the transform learning scheme can improve the sparse representation ability. An efficient block coordinate descent (BCD) algorithm is developed to solve the proposed model. Extensive experiments, including simulated and real HSI datasets, indicate the superiority of the proposed TLNLGS method over the state-of-the-art HSI denoising approaches.

Fusing a pair of low-spatial-resolution hyperspectral image (LR-HSI) and high-spatial-resolution multispectral image (HR-MSI) has been regarded as an effective and economical strategy to achieve HR-HSI, which is essential to many applications. Among existing fusion models, the tensor ring (TR) decomposition-based model has attracted rising attention due to its superiority in approximating high-dimensional data compared to other traditional matrix/tensor decomposition models. Unlike directly estimating HR-HSI in traditional models, the TR fusion model translates the fusion procedure into an estimate of the TR factor of HR-HSI, which can efficiently capture the spatial–spectral correlation of HR-HSI. Although the spatial–spectral correlation has been preserved well by TR decomposition, the spatial–spectral continuity of HR-HSI is ignored in existing TR decomposition models, sometimes resulting in poor quality of reconstructed images. In this article, we introduce a factor smoothed regularization for TR decomposition to capture the spatial–spectral continuity of HR-HSI. As a result, our proposed model is calledfactor smoothed TR decompositionmodel, dubbedFSTRD. In order to solve the suggested model, we develop an efficient proximal alternating minimization algorithm. A series of experiments on four synthetic datasets and one real-world dataset show that the quality of reconstructed images can be significantly improved by the introduced factor smoothed regularization, and thus, the suggested method yields the best performance by comparing it to state-of-the-art methods.

This article proposes a new denoising method for hyperspectral images (HSIs) corrupted by mixtures (in a statistical sense) of stripe noise, Gaussian noise, and impulsive noise. The proposed method has three distinctive features: 1) it exploits the intrinsic characteristics of HSIs, namely, low-rank and self-similarity; 2) the observation noise is assumed to be additive and modeled by a mixture of Gaussian (MoG) densities; 3) the inference is performed with an expectation maximization (EM) algorithm, which, in addition to the clean HSI, also estimates the mixture parameters (posterior probability of each mode and variances). Comparisons of the proposed method with state-of-the-art algorithms provide experimental evidence of the effectiveness of the proposed denoising algorithm. A MATLAB demo of this work will be available athttps://github.com/TaiXiangJiangfor the sake of reproducibility.

The existing nonblind cloud and cloud shadow (cloud/shadow) removal methods for remote sensing (RS) images are based on the assumption that cloud/shadow masks are accurately given. Since the masks are usually manually labeled or detected by cloud detection methods, whose accuracy cannot be well guaranteed, the cloud/shadow removal effect may be affected. In this article, we suggest a robust thick cloud/shadow removal (RTCR) method that meets the problem with an inaccurate mask. To faithfully reconstruct the multitemporal information, a coupled tensor factorization is used to explore the relationship between the abundances of the multitemporal images in the same scene. Moreover, an efficient algorithm is developed to solve the proposed model based on the augmented Lagrange multiplier method. The experimental results under accurate masks and inaccurate masks demonstrate its robustness and superiority for thick cloud/shadow removal.

Pansharpening refers to a spatio-spectral fusion of a lower spatial resolution multispectral (MS) image with a high spatial resolution panchromatic image, aiming at obtaining an image with a corresponding high resolution both in the domains. In this article, we propose a generic fusion framework that is able to weightedly combine variational optimization (VO) with deep learning (DL) for the task of pansharpening, where these crucial weights directly determining the relative contribution of DL to each pixel are estimated adaptively. This framework can benefit from both VO and DL approaches, e.g., the good modeling explanation and data generalization of a VO approach with the high accuracy of a DL technique thanks to massive data training. The proposed method can be divided into three parts: 1) for the VO modeling, a general details injection term inspired by the classical multiresolution analysis is proposed as a spatial fidelity term and a spectral fidelity employing the MS sensor’s modulation transfer functions is also incorporated; 2) for the DL injection, a weighted regularization term is designed to introduce deep learning into the variational model; and 3) the final convex optimization problem is efficiently solved by the designed alternating direction method of multipliers. Extensive experiments both at reduced and full-resolution demonstrate that the proposed method outperforms recent state-of-the-art pansharpening methods, especially showing a higher accuracy and a significant generalization ability.

Hyperspectral image (HSI) denoising is a fundamental task in remote sensing image processing, which is helpful for HSI subsequent applications, such as unmixing and classification. Thanks to the powerful representation ability of untrained deep neural networks, deep image prior (DIP)-based methods achieve tremendous successes in image processing (e.g., denoising and inpainting). However, DIP-based methods neglect the tensor low-rank prior of the underlying HSI which will be beneficial to capturing the global structure of the underlying HSI. To address this issue, we propose a novel model for HSI denoising, which can simultaneously take respective advantages of the tensor low-rank prior and the deep spatial-spectral prior. The tensor low-rank prior leads to a better global structure and the deep spatial-spectral prior is complementary to preserve better local details. On the one hand, we adopt low-rank tensor ring (TR) decomposition to characterize the tensor low-rank prior and capture the global structure of the underlying HSI. On the other hand, we employ untrained deep neural networks to flexibly represent the deep spatial-spectral prior and capture the local details of the underlying HSI. To solve the proposed model, we develop an efficient alternating minimization algorithm. Experimental results on simulated and real data validate the advantages of the proposed model in HSI denoising. Compared with state-of-the-art HSI denoising methods, the proposed method preserves better local details and the global structure of the underlying HSI.

Pansharpening is related to the fusion of a low spatial resolution multispectral (MS) image retaining an abundant spectral content and a high spatial resolution panchromatic (PAN) image to obtain a product with both the abundant spectral content of the former and the high spatial resolution of the latter. Many previous studies are only focused on the global or local relationship between the PAN image and the corresponding high-resolution multispectral (HRMS) image. However, we found that the relationship between PAN and HRMS images in the gradient domain can be better explored through the image context. In this article, we propose context-aware details injection fidelity (CDIF) with adaptive coefficients estimation, which can fully explore the complicated relationship between the PAN image and the HRMS image in the gradient domain. More specifically, we apply a clustering method to divide the pixels of an image into different context-based regions. Afterward, the adaptive coefficients are estimated by using a regression-based method for each region. The CDIF is effective in extracting the main features from the two inputs to be fused. In addition, we integrate the CDIF with a conventional fidelity term and a total variation regularization to formulate a novel variational pansharpening model that is solved by designing an algorithm based on the alternating direction method of multiplier (ADMM) framework. Qualitative and quantitative assessments on different datasets support the effectiveness and robustness of the proposed method. The code is available athttps://github.com/liangjiandeng/CDIF.

Hyperspectral image super-resolution (HSI-SR) can be achieved by fusing a paired multispectral image (MSI) and hyperspectral image (HSI), which is a prevalent strategy. But, how to precisely reconstruct the high spatial resolution hyperspectral image (HR-HSI) by fusion technology is a challenging issue. In this paper, we propose an iterative regularization method based on tensor subspace representation (IR-TenSR) for MSI-HSI fusion, thus HSI-SR. First, we propose a tensor subspace representation (TenSR)-based regularization model that integrates the global spectral-spatial low-rank and the nonlocal self-similarity priors of HR-HSI. These two priors have been proven effective, but previous HSI-SR works cannot simultaneously exploit them. Subsequently, we design an iterative regularization procedure to utilize the residual information of acquired low-resolution images, which are ignored in other works that produce suboptimal results. Finally, we develop an effective algorithm based on the proximal alternating minimization method to solve the TenSR-regularization model. With that, we obtain the iterative regularization algorithm. Experiments implemented on the simulated and real datasets illustrate the advantages of the proposed IR-TenSR compared with state-of-the-art fusion approaches. The code is available at https://github.com/liangjiandeng/IR-TenSR.

Completing missing entries in multidimensional visual data is a typical ill-posed problem that requires appropriate exploitation of prior information of the underlying data. Commonly used priors can be roughly categorized into three classes: global tensor low-rankness, local properties, and nonlocal self-similarity (NSS); most existing works utilize one or two of them to implement completion. Naturally, there arises an interesting question: can one concurrently make use of multiple priors in a unified way, such that they can collaborate with each other to achieve better performance? This work gives a positive answer by formulating a novel tensor completion framework which can simultaneously take advantage of the global-local-nonlocal priors. In the proposed framework, the tensor train (TT) rank is adopted to characterize the global correlation; meanwhile, two Plug-and-Play (PnP) denoisers, including a convolutional neural network (CNN) denoiser and the color block-matching and 3 D filtering (CBM3D) denoiser, are incorporated to preserve local details and exploit NSS, respectively. Then, we design a proximal alternating minimization algorithm to efficiently solve this model under the PnP framework. Under mild conditions, we establish the convergence guarantee of the proposed algorithm. Extensive experiments show that these priors organically benefit from each other to achieve state-of-the-art performance both quantitatively and qualitatively.

Hyperspectral images (HSIs) are of crucial importance in order to better understand features from a large number of spectral channels. Restricted by its inner imaging mechanism, the spatial resolution is often limited for HSIs. To alleviate this issue, in this work, we propose a simple and efficient architecture of deep convolutional neural networks to fuse a low-resolution HSI (LR-HSI) and a high-resolution multispectral image (HR-MSI), yielding a high-resolution HSI (HR-HSI). The network is designed to preserve both spatial and spectral information thanks to a new architecture based on: 1) the use of the LR-HSI at the HR-MSI's scale to get an output with satisfied spectral preservation and 2) the application of the attention and pixelShuffle modules to extract information, aiming to output high-quality spatial details. Finally, a plain mean squared error loss function is used to measure the performance during the training. Extensive experiments demonstrate that the proposed network architecture achieves the best performance (both qualitatively and quantitatively) compared with recent state-of-the-art HSI super-resolution approaches. Moreover, other significant advantages can be pointed out by the use of the proposed approach, such as a better network generalization ability, a limited computational burden, and the robustness with respect to the number of training samples. Please find the source code and pretrained models from https://liangjiandeng.github.io/Projects_Res/HSRnet_2021tnnls.html.

The popular tensor train (TT) and tensor ring (TR) decompositions have achieved promising results in science and engineering. However, TT and TR decompositions only establish an operation between adjacent two factors and are highly sensitive to the permutation of tensor modes, leading to an inadequate and inflexible representation. In this paper, we propose a generalized tensor decomposition, which decomposes an Nth-order tensor into a set of Nth-order factors and establishes an operation between any two factors. Since it can be graphically interpreted as a fully-connected network, we named it fully-connected tensor network (FCTN) decomposition. The superiorities of the FCTN decomposition lie in the outstanding capability for characterizing adequately the intrinsic correlations between any two modes of tensors and the essential invariance for transposition. Furthermore, we employ the FCTN decomposition to one representative task, i.e., tensor completion, and develop an efficient solving algorithm based on proximal alternating minimization. Theoretically, we prove the convergence of the developed algorithm, i.e., the sequence obtained by it globally converges to a critical point. Experimental results substantiate that the proposed method compares favorably to the state-of-the-art methods based on other tensor decompositions.

Deep Convolution Neural Networks have been adopted for pansharpening and achieved state-of-the-art performance. However, most of the existing works mainly focus on single-scale feature fusion, which leads to failure in fully considering relationships of information between high-level semantics and low-level features, despite the network is deep enough. In this paper, we propose a dynamic cross feature fusion network (DCFNet) for pansharpening. Specifically, DCFNet contains multiple parallel branches, including a high-resolution branch served as the backbone, and the low-resolution branches progressively supplemented into the backbone. Thus our DCFNet can represent the overall information well. In order to enhance the relationships of inter-branches, dynamic cross feature transfers are embedded into multiple branches to obtain high-resolution representations. Then contextualized features will be learned to improve the fusion of information. Experimental results indicate that DCFNet significantly outperforms the prior arts in both quantitative indicators and visual qualities.

The hyperspectral unmixing (HU) aims at estimating the spectral signatures of endmembers (or materials) and their corresponding abundance maps of the hyperspectral image (HSI). In this work, we treat the HSI as an 3D cube and propose a new tensor-based HU method. We decompose an HSI data as the sum of several multilinear rank-(Lr, Lr, 1) terms (or LL1 model). Based on the fact that the latent factors of LL1 model are physical meaningful (i.e., abundance maps and spectral signatures), we build a nonnegative tensor decomposition optimization model with the low-rank constraint and impose an implicit regularizer to exploit the nonlocal self-similarity prior of abundance maps-whose related subproblem can be easily solved under the plug-and-play framework. We develop an alternating direction method of multipliers algorithm to solve the proposed model. Numerical experiments demonstrate the effectiveness of our algorithm.

Sparse hyperspectral unmixing has attracted much attention in recent decades. Recently, the low-rank representation provides a new perspective for spatial correlation and the weighted nuclear norm regularization has been well studied to enhance the low-rankness of the abundance matrix. However, the commonly used weights only depend on respective singular values, ignoring other singular values' information. In this paper, we propose a new weighting scheme for the weighed nuclear norm to further enhance the sparsity of the singular values of the abundance matrix. The proposed weight for each singular value considers information of all singular values, instead of particular singular value only. Then we refine two sparsity and low-rankness based unmixing algorithms. Simulated and real-data experiments demonstrate the effectiveness of the resulting unmixing algorithms.

For multi-temporal remote sensing (RS) images, the distribution of surface materials is constant concerning time and the same material shows different spectral features at different times. Decomposing the image at each time into an abundance tensor and temporal features, there is a strong similarity between abundance tensors of all time. Based on this observation, we suggest a blind thick cloud/shadow removal model, which exploits the sparsity of the cloud component and the similarity between abundance tensors, achieving both cloud detection and multi-temporal information restoration. Moreover, a mask refinement strategy is designed to pursue the optimal cloud/shadow mask. We develop an efficient algorithm to solve the proposed model based on the augmented Lagrange multiplier method. The results of simulated experiments in different scenarios verify the superiority of the proposed method for thick cloud/shadow removal.

Recently, unsupervised deep learning-based methods have shown an empirical success in hyperspectral images (HSIs) denoising, profiting from the strong representation ability of convolutional neural networks. However, these methods only can describe the local structure of the spatial dimension, which is restricted to very limited local receptive fields. To overcome this difficulty, a novel HSIs denoising model based on the deep image prior (DIP) framework is proposed by adding a tensor ring (TR) decomposition, which can enlarge the receptive field of the spatial dimension and capture global spectral correlation simultaneously. Unlike the previous DIP framework that directly minimizes the objective function, we develop an algorithm based on proximal alternating minimization to decouple the model into the classic DIP framework and TR cores least-squares problems, which are easy to solve. Experimental results verify that the proposed DIP- TR compares favorably with compared methods in terms of quality metrics and visual performance.

Hyperspectral unmixing (HU), estimating endmembers and the corresponding abundances, is crucial for the development of hyperspectral images (HSIs). To improve the unmixing performance, various spatial regularizers are imposed on the abundance matrix. Note that endmember information is also important for HU, especially when the spectral signature in HSIs are highly correlated. In this paper, we investigate information from both endmembers and abundances and propose an endmember constraint non-negative tensor factorization via total variation (EC-NTF-TV) for HU. For estimating end-members, we introduce an endmember constraint to alleviate the spectral signatures' high correlation. In addition, we adopt the TV regularization to exploit the spatial correlation in abundance maps. Finally, we solve the proposed model under the augmented multiplicative update framework. Both synthetic and real hyperspectral data experiments demonstrate the effectiveness of the proposed algorithm.

The aim of hyperspectral image super-resolution (HSI-SR) is to produce high spatial resolution hyperspectral image (HR - HSI) by exploiting the available high spatial resolution multispectral image (HR-MSI) and low spatial resolution hyperspectral image (LR-HSI). In this work, we develop a novel matrix factorization (MF)-based HSI -SR way, which formulates the HSI -SR problem as estimating the spectral dictionary from the observed LR - HSI and the coefficient matrix from both the observed HR-MSI and LR-HSI. Specifically, we first estimate the spectral dictionary from the observed LR - HSI by the dictionary learning algorithm with redundancy assumption. Moreover, based on the superpixel segmentation technology used in the observed HR-MSI, the coefficient vectors are grouped. By concatenating the joint-sparse, nonlocallow-rank, and nonnegative priors of the grouped coefficient vectors, we develop a novel coefficient matrix estimation variational model, which fully explores the nonlocal self-similarity of the desired HR-HSI. The proposed coefficient matrix estimation model is solved under the alternating direction method of multipliers (ADMM) framework. Experimental results prove the superiority of the proposed way from the quantitative and qualitative analysis.

Sparse unmixing is an important technique for analyzing and processing hyperspectral images (HSIs). Simultaneously exploiting spatial correlation and sparsity improves substantially abundance estimation accuracy. In this article, we propose to exploit nonlocal spatial information in the HSI for the sparse unmixing problem. Specifically, we first group similar patches in the HSI, and then unmix each group by imposing simultaneous a low-rank constraint and joint sparsity in the corresponding third-order abundance tensor. To this end, we build an unmixing model with a mixed regularization term consisting of the sum of the weighted tensor trace norm and the weighted tensor$\ell _{2,1}$-norm of the abundance tensor. The proposed model is solved under the alternating direction method of multipliers framework. We term the developed algorithm as the nonlocal tensor-based sparse unmixing algorithm. The effectiveness of the proposed algorithm is illustrated in experiments with both simulated and real hyperspectral data sets.

In hyperspectral image (HSI) denoising, subspace-based denoising methods can reduce the computational complexity of the denoising algorithm. However, the existing matrix subspaces, which are generated by the unfolding matrix of the HSI tensor, cannot completely represent a tensor since the unfolding operation will destroy the tensor structure. To overcome this, we design a novel basis tensor that is directly learned from the original tensor and present a tensor subspace representation (TenSR), which is a more authentic representation for delivering the intrinsic structure of the tensor than a matrix subspace representation. Equipped with the TenSR, we then propose a TenSR-based HSI denoising (TenSRDe) model, which simultaneously considers the low-tubal rankness of the HSI tensor and the nonlocal self-similarity of the coefficient tensor. Moreover, we develop an efficient proximal alternating minimization (PAM) algorithm to solve the proposed nonconvex model and theoretically prove that the algorithm globally converges to a critical point. Experiments implemented on simulated and real data sets substantiate the denoising effect and efficiency of the proposed method.

This paper addresses the tensor completion problem, which aims to recover missing information of multi-dimensional images. How to represent a low-rank structure embedded in the underlying data is the key issue in tensor completion. In this work, we suggest a novel low-rank tensor representation based on coupled transform, which fully exploits the spatial multi-scale nature and redundancy in spatial and spectral/temporal dimensions, leading to a better low tensor multi-rank approximation. More precisely, this representation is achieved by using two-dimensional framelet transform for the two spatial dimensions, one/two-dimensional Fourier transform for the temporal/spectral dimension, and then Karhunen-Loéve transform (via singular value decomposition) for the transformed tensor. Based on this low-rank tensor representation, we formulate a novel low-rank tensor completion model for recovering missing information in multi-dimensional visual data, which leads to a convex optimization problem. To tackle the proposed model, we develop the alternating directional method of multipliers (ADMM) algorithm tailored for the structured optimization problem. Numerical examples on color images, multispectral images, and videos illustrate that the proposed method outperforms many state-of-the-art methods in qualitative and quantitative aspects.

Recently emerged deep learning methods have achieved great success in single image rain streaks removal. However, existing methods ignore an essential factor in the rain streaks generation mechanism, i.e., the motion blur leading to the line pattern appearances. Thus, they generally produce overderaining or underderaining results. In this article, inspired by the generation mechanism, we propose a novel rain streaks removal framework using a kernel-guided convolutional neural network (KGCNN), achieving state-of-the-art performance with a simple network architecture. More precisely, our framework consists of three steps. First, we learn the motion blur kernel by a plain neural network, termed parameter network, from the detail layer of a rainy patch. Then, we stretch the learned motion blur kernel into a degradation map with the same spatial size as the rainy patch. Finally, we use the stretched degradation map together with the detail patches to train a deraining network with a typical ResNet architecture, which produces the rain streaks with the guidance of the learned motion blur kernel. Experiments conducted on extensive synthetic and real data demonstrate the effectiveness of the proposed KGCNN, in terms of rain streaks removal and image detail preservation.

Schizophrenia (SZ) is a complex disease. Single nucleotide polymorphism (SNP), brain activity measured by functional magnetic resonance imaging (fMRI) and DNA methylation are all important biomarkers that can be used for the study of SZ. To our knowledge, there has been little effort to combine these three datasets together. In this study, we propose a group sparse joint nonnegative matrix factorization (GSJNMF) model to integrate SNP, fMRI, and DNA methylation for the identification of multi-dimensional modules associated with SZ, which can be used to study regulatory mechanisms underlying SZ at multiple levels. The proposed GSJNMF model projects multiple types of data onto a common feature space, in which heterogeneous variables with large coefficients on the same projected bases are used to identify multi-dimensional modules. We also incorporate group structure information available from each dataset. The genomic factors in such modules have significant correlations or functional associations with several brain activities. At the end, we have applied the method to the analysis of real data collected from the Mind Clinical Imaging Consortium (MCIC) for the study of SZ and identified significant biomarkers. These biomarkers were further used to discover genes and corresponding brain regions, which were confirmed to be significantly associated with SZ.

Mixed noise (such as Gaussian, impulse, stripe, and deadline noises) contamination is a common phenomenon in hyperspectral imagery (HSI), greatly degrading visual quality and affecting subsequent processing accuracy. By encoding sparse prior to the spatial or spectral difference images, total variation (TV) regularization is an efficient tool for removing the noises. However, the previous TV term cannot maintain the shared group sparsity pattern of the spatial difference images of different spectral bands. To address this issue, this article proposes a group sparsity regularization of the spatial difference images for HSI restoration. Instead of using ℓ1or ℓ2-norm (sparsity) on the difference image itself, we introduce a weighted ℓ2,1norm to constrain the spatial difference image cube, efficiently exploring the shared group sparse pattern. Moreover, we employ the well-known low-rank Tucker decomposition to capture the global spatial-spectral correlation from three HSI dimensions. To summarize, a weighted group sparsity-regularized low-rank tensor decomposition (LRTDGS) method is presented for HSI restoration. An efficient augmented Lagrange multiplier algorithm is employed to solve the LRTDGS model. The superiority of this method for HSI restoration is demonstrated by a series of experimental results from both simulated and real data, as compared with the other state-of-the-art TV-regularized low-rank matrix/tensor decomposition methods.

Hyperspectral image (HSI) denoising is a fundamental problem in remote sensing and image processing. Recently, nonlocal low-rank tensor approximation-based denoising methods have attracted much attention due to their advantage of being capable of fully exploiting the nonlocal self-similarity and global spectral correlation. Existing nonlocal low-rank tensor approximation methods were mainly based on two common decomposition [Tucker or CANDECOMP/PARAFAC (CP)] methods and achieved the state-of-the-art results, but they are subject to certain issues and do not produce the best approximation for a tensor. For example, the number of parameters for Tucker decomposition increases exponentially according to its dimensions, and CP decomposition cannot better preserve the intrinsic correlation of the HSI. In this article, a novel nonlocal tensor-ring (TR) approximation is proposed for HSI denoising by using TR decomposition to explore the nonlocal self-similarity and global spectral correlation simultaneously. TR decomposition approximates a high-order tensor as a sequence of cyclically contracted third-order tensors, which has strong ability to explore these two intrinsic priors and to improve the HSI denoising results. Moreover, an efficient proximal alternating minimization algorithm is developed to optimize the proposed TR decomposition model efficiently. Extensive experiments on three simulated data sets under several noise levels and two real data sets verify that the proposed TR model provides better HSI denoising results than several state-of-the-art methods in terms of quantitative and visual performance evaluations.

As a preprocessing step, hyperspectral image (HSI) restoration plays a critical role in many subsequent applications. Recently, based on the framework of subspace representation and low-rank matrix/tensor factorization (LRMF/LRTF), many single-factor-regularized methods add various regularizations on the spatial factor to characterize its spatial prior knowledge. However, these methods neglect the common characteristics among different bands and the spectral continuity of HSIs. To tackle this issue, this article establishes a bridge between the factor-based regularization and the HSI priors and proposes a double-factor-regularized LRTF model for HSI mixed noise removal. The proposed model employs LRTF to characterize the spectral global low rankness, introduces a weighted group sparsity constraint on the spatial difference images (SpatDIs) of the spatial factor to promote the group sparsity in the SpatDIs of HSIs, and suggests a continuity constraint on the spectral factor to promote the spectral continuity of HSIs. Moreover, we develop a proximal alternating minimization-based algorithm to solve the proposed model. Extensive experiments conducted on the simulated and real HSIs demonstrate that the proposed method has superior performance on mixed noise removal compared with the state-of-the-art methods based on subspace representation, noise modeling, and LRMF/LRTF.

The tensor tubal rank, defined based on the tensor singular value decomposition (t-SVD), has obtained promising results in hyperspectral image (HSI) denoising. However, the framework of the t-SVD lacks flexibility for handling different correlations along different modes of HSIs, leading to suboptimal denoising performance. This article mainly makes three contributions. First, we introduce a new tensor rank named tensor fibered rank by generalizing the t-SVD to the mode-k t-SVD, to achieve a more flexible and accurate HSI characterization. Since directly minimizing the fibered rank is NP-hard, we suggest a three-directional tensor nuclear norm (3DTNN) and a three-directional log-based tensor nuclear norm (3DLogTNN) as its convex and nonconvex relaxation to provide an efficient numerical solution, respectively. Second, we propose a fibered rank minimization model for HSI mixed noise removal, in which the underlying HSI is modeled as a low-fibered-rank component. Third, we develop an efficient alternating direction method of multipliers (ADMMs)-based algorithm to solve the proposed model, especially, each subproblem within ADMM is proven to have a closed-form solution, although 3DLogTNN is nonconvex. Extensive experimental results demonstrate that the proposed method has superior denoising performance, as compared with the state-of-the-art competing methods on low-rank matrix/tensor approximation and noise modeling.

Hyperspectral image compressive sensing reconstruction (HSI-CSR) can largely reduce the high expense and low efficiency of transmitting HSI to ground stations by storing a few compressive measurements, but how to precisely reconstruct the HSI from a few compressive measurements is a challenging issue. It has been proven that considering the global spectral correlation, spatial structure, and nonlocal self-similarity priors of HSI can achieve satisfactory reconstruction performances. However, most of the existing methods cannot simultaneously capture the mentioned priors and directly design the regularization term to the HSI. In this article, we propose a novel subspace-based nonlocal tensor ring decomposition method (SNLTR) for HSI-CSR. Instead of designing the regularization of the low-rank approximation to the HSI, we assume that the HSI lies in a low-dimensional subspace. Moreover, to explore the nonlocal self-similarity and preserve the spatial structure of HSI, we introduce a nonlocal tensor ring decomposition strategy to constrain the related coefficient image, which can decrease the computational cost compared to the methods that directly employ the nonlocal regularization to HSI. Finally, a well-known alternating minimization method is designed to efficiently solve the proposed SNLTR. Extensive experimental results demonstrate that our SNLTR method can significantly outperform existing approaches for HSI-CSR.

The main aim of this paper is to develop a framelet representation of the tensor nuclear norm for third-order tensor recovery. In the literature, the tensor nuclear norm can be computed by using tensor singular value decomposition based on the discrete Fourier transform matrix, and tensor completion can be performed by the minimization of the tensor nuclear norm which is the relaxation of the sum of matrix ranks from all Fourier transformed matrix frontal slices. These Fourier transformed matrix frontal slices are obtained by applying the discrete Fourier transform on the tubes of the original tensor. In this paper, we propose to employ the framelet representation of each tube so that a framelet transformed tensor can be constructed. Because of framelet basis redundancy, the representation of each tube is sparsely represented. When the matrix slices of the original tensor are highly correlated, we expect the corresponding sum of matrix ranks from all framelet transformed matrix frontal slices would be small, and the resulting tensor completion can be performed much better. The proposed minimization model is convex and global minimizers can be obtained. Numerical results on several types of multi-dimensional data (videos, multispectral images, and magnetic resonance imaging data) have tested and shown that the proposed method outperformed the other testing methods.

Images captured in rainy conditions are often corrupted by unexpected rain streaks, which severely degrade the performance of subsequent processes in outdoor computer vision systems. In this paper, we exploit the directional smoothness of rain streaks for the single-image rain streaks removal and propose a convex model that uses the directional total variation (DTV) to characterize the smoothness of rain streaks in arbitrary orientations. The proposed model consists of four terms: the fidelity term, the ℓ1norm for the sparsity of rain streaks, and two DTV regularization terms for the directional smoothness and the piecewise smoothness of rain streaks and rain-free backgrounds, respectively. To solve the proposed model, we develop an efficient algorithm based on the alternating direction method of multipliers (ADMM) framework. Extensive experimental results on both synthetic and real rainy images show that our method outperforms the recent state-of-the-art methods visually and quantitatively.

Total variation (TV) regularization has been widely used in the hyperspectral image (HSI) mixed noise removal problem by utilizing ℓ1-norm to constrain the spatial difference image and promote the piecewise smooth structure. Unfortunately, it cannot depict the group sparse structure of spatial difference image along the spectral dimension. This paper proposes a new HSI restoration method using weighted group sparsity regularized low-rank tensor decomposition (LRTDGS). Specifically, we use a weighted group sparsity regularization which is denoted by ℓ2,1-norm to explore the group structure of spatial difference image along the spectral dimension. Moreover, the spatial-spectral correlation from three directions of HSI is depicted by low-rank Tucker decomposition. We use efficient augmented Lagrange multiplier method to optimize the proposed LRTDGS model, and a series of experimental results are presented to demonstrate the effectiveness of the proposed method.

This paper proposes a spatial-spectral total variation (TV) regularized low-rank sparsity decomposition model for blind cloud and cloud shadow (cloud/shadow) detection and removal of multitemporal remote sensing imagery. Our concept is to decompose the contaminated image into the surface-reflected component and the cloud/shadow component. Low-rank regularization is utilized to model the spectral-temporal correlation of the surface-reflected component, meanwhile, the `1-norm and spatial-spectral total variation regularization is employed to describe the sparse prior and spatial-spectral continuity of the cloud/shadow component. To better preserve the information in cloud/shadow-free areas, the cloud/shadow detection results obtained as a by-product of our method are used to guide the information compensation from the original contaminated images. Several experiments are presented to demonstrate the effectiveness of the proposed method.

In this paper, we mainly focus on a quite challenging denoising problem in remote sensing images, which is to simultaneously remove Gaussian noise and sparse noise that mainly include stripes and salt-pepper noise. We propose a convex unidirectional sparse model based on mode-3 tensor modeling to remove the mixture noise. A proximal alternating direction method of multipliers (ADMM) based algorithm is designed to effectively solve the given minimization model. Comparing with some recent state-of-the-art denoising methods, the proposed method shows the best performance from visual and quantitative aspects.

In this paper, a pan-sharpening approach based on RoG filtering is proposed. This approach follows the framework of classic methods of pan-sharpening, i.e., component substitution and multi-resolution analysis. The filtering technique based on Relativity-of-Gaussian (RoG) regularization is first used in the process of upsampling the original multi-spectral image, and then in the detail extraction phase to obtain spatial details from the panchromatic image. Experiments on datasets acquired by Quickbird and IKONOS demonstrate that the proposed approach obtains competitive performance comparing with several popular pan-sharpening methods.

In this paper, we propose a new tensor rank, named fibered rank, by generalizing the tensor singular value decomposition (t-SVD) to the mode-k t-SVD. It factorizes a three-way tensor into two mode-k orthogonal tensors and a mode-k diagonal tensor. To efficiently remove mixed noise and finely preserve the structure information, we propose a novel hyperspectral image denoising model based on the three-directional tensor nuclear norm (3DTNN), which is a convex relaxation of the fibered rank. An efficient alternating direction method of multipliers (ADMM)-based algorithm is developed to solve the proposed model. Experimental results demonstrate the superiority of the proposed method over the compared ones.

Hyperspectral unmixing has attracted much attention in recent years. Single sparse unmixing assumes that a pixel in a hyperspectral image consists of a relatively small number of spectral signatures from large, ever-growing, and available spectral libraries. Joint-sparsity (or row-sparsity) model typically enforces all pixels in a neighborhood to share the same set of spectral signatures. The two sparse models are widely used in the literature. In this paper, we propose a joint-sparsity-blocks model for abundance estimation problem. Namely, the abundance matrix of size m × n is partitioned to have one row block and s column blocks and each column block itself is joint-sparse. It generalizes both the single (i.e., s = n) and the joint (i.e., s = 1) sparsities. Moreover, concatenating the proposed joint-sparsity-blocks structure and low rankness assumption on the abundance coefficients, we develop a new algorithm called joint-sparseblocks and low-rank unmixing. In particular, for the joint-sparseblocks regression problem, we develop a two-level reweighting strategy to enhance the sparsity along the rows within each block. Simulated and real-data experiments demonstrate the effectiveness of the proposed algorithm.

On May 31, 2003, the scan line corrector (SLC) of the Enhanced Thematic Mapper Plus (ETM+) on-board the Landsat-7 satellite failed, resulting in strips of data lost in all ETM+ images acquired since then. In this paper, we proposed a novel inpainting algorithm for recovering the ETM+ SLC-off images. The two slopes of the boundaries of each missing stripe were extracted through the Hough transform, ignoring the slope of the edge of the strip that overlaps the edge of the image. An adaptive dictionary was then developed and trained using ETM+ SLC-on images acquired before May 31, 2003 so that the physical characteristics and geometric features of the ground coverage of the data-missing strips can be considered during recovery. To make the algorithm computationally efficient, data-missing strips were repaired along their slope directions by using the logdet $\left ({\cdot }\right)$ low-rank nonconvex model along with the dictionary. The algorithm was tested using the simulated ETM+ SLC-off images created from a multiband ETM+ SLC-on image file and compared to the high accuracy low-rank tensor completion (HaLRTC), logDet, and tensor nuclear norm (TNN) algorithms. The results show that the ETM+ images restored using the new algorithm have lower RMSE, higher PSNR and structure similarity (SSIM) values, and better visualization. These results indicate that the new algorithm performs better than the other three algorithms and can efficiently and accurately restore the data-missing stripes.

Rain streaks removal is an important issue in outdoor vision systems and has recently been investigated extensively. In this paper, we propose a novel video rain streak removal approach FastDeRain, which fully considers the discriminative characteristics of rain streaks and the clean video in the gradient domain. Specifically, on the one hand, rain streaks are sparse and smooth along the direction of the raindrops, whereas on the other hand, clean videos exhibit piecewise smoothness along the rain-perpendicular direction and continuity along the temporal direction. Theses smoothness and continuity results in the sparse distribution in the different directional gradient domain, respectively. Thus, we minimize 1) the ℓ1 norm to enhance the sparsity of the underlying rain streaks, 2) two ℓ1 norm of unidirectional Total Variation (TV) regularizers to guarantee the anisotropic spatial smoothness, and 3) an ℓ1 norm of the time-directional difference operator to characterize the temporal continuity. A split augmented Lagrangian shrinkage algorithm (SALSA) based algorithm is designed to solve the proposed minimization model. Experiments conducted on synthetic and real data demonstrate the effectiveness and efficiency of the proposed method. According to comprehensive quantitative performance measures, our approach outperforms other state-of-the-art methods, especially on account of the running time. The code of FastDeRain can be downloaded at https://github.com/TaiXiangJiang/FastDeRain.

This paper proposes a new denoising method for hyperspectral images (HSIs) corrupted by mixtures (in a statistical sense) of stripe noise, Gaussian noise, and impulsive noise. The proposed method has three distinctive features: 1) it exploits the intrinsic characteristics of HSIs, namely, low-rank and self-similarity; 2) the observation noise is assumed to be additive and modeled by a mixture of Gaussian (MoG) densities; 3) the inference is performed with an expectation maximization (EM) algorithm, which, in addition to the clean HSI, also estimates the mixture parameters (posterior probability of each mode and variances). Comparisons of the proposed method with state-of-the-art algorithms provide experimental evidence of the effectiveness of the proposed denoising algorithm.

Remotely sensed images may contain some missing areas because of poor weather conditions and sensor failure. Information of those areas may play an important role in the interpretation of multitemporal remotely sensed data. This paper aims at reconstructing the missing information by a nonlocal low-rank tensor completion method. First, nonlocal correlations in the spatial domain are taken into account by searching and grouping similar image patches in a large search window. Then, low rankness of the identified fourth-order tensor groups is promoted to consider their correlations in spatial, spectral, and temporal domains, while reconstructing the underlying patterns. Experimental results on simulated and real data demonstrate that the proposed method is effective both qualitatively and quantitatively. In addition, the proposed method is computationally efficient compared with other patch-based methods such as the recently proposed patch matching-based multitemporal group sparse representation method.

Rain streaks removal is an important issue of the outdoor vision system and has been recently investigated extensively. In this paper, we propose a novel tensor based video rain streaks removal approach by fully considering the discriminatively intrinsic characteristics of rain streaks and clean videos, which needs neither rain detection nor time-consuming dictionary learning stage. In specific, on the one hand, rain streaks are sparse and smooth along the raindrops direction, and on the other hand, the clean videos possess smoothness along the rain-perpendicular direction and global and local correlation along time direction. We use the l1 norm to enhance the sparsity of the underlying rain, two unidirectional Total Variation (TV) regularizers to guarantee the different discriminative smoothness, and a tensor nuclear norm and a time directional difference operator to characterize the exclusive correlation of the clean video along time. Alternation direction method of multipliers (ADMM) is employed to solve the proposed concise tensor based convex model. Experiments implemented on synthetic and real data substantiate the effectiveness and efficiency of the proposed method. Under comprehensive quantitative performance measures, our approach outperforms other state-of-the-art methods.

Schizophrenia (SZ) is a complex disease caused by a lot genetic variants, epigenetic and brain region abnormalities. In this study, we adopted a joint nonnegative matrix factorization method to integrate three datasets including single nucleotide polymorphism (SNP), brain activity measured by functional magnetic resonance imaging (fMRI) and DNA Methylation to identify multi-dimensional modules associated with SZ. They are then used to study the coordination between regulatory mechanisms at multiple levels. This method projects multiple types of data onto a common feature space, in which heterogeneous variables with large coefficients on the same projected bases form a multi-dimensional module. The genomic factors in such modules have significant correlations and likely functional associations with brain activities. We applied this method to the real data analysis and identified multi-dimensional modules including SNP, fMRI and DNA methylation sites. These selected biomarkers were finally used to identify genes and voxels, which were confirmed to be significantly associated with SZ.

This paper commits to remove the stripe noise to enhance the visual quality of remote sensing images, in the meanwhile preserves image details of stripe-free regions. Instead of solving the underlying image as most of researches, we propose a non-convex l0model for remote sensing image destriping by taking full consideration of the intrinsically directional and structural priors of stripe noise. Moreover, the proposed non-convex model can be solved by the proximal alternating direction method of multipliers (PADMM) method which theoretically guarantees converging to a KKT point. Extensively experimental results on simulated and real data demonstrate that the proposed method outperforms recent state-of-the-art destriping methods, both visually and quantitatively.

We propose a novel image inpainting model that can effectively estimate missing pixels in an observed image. The latent image is characterized by a group-based low-rank prior, which assumes that a group of vectorized similar image patches can be well approximated by a low-rank matrix. We enforce the low-rankness of each group by penalizing a truncated difference of the l1and the l2norms of its singular values, which achieves a close approximation to the matrix rank. We apply a difference of convex algorithm (DCA) to solve the proposed model efficiently. Our method is validated on filling missing blocks and randomly missing pixels, with superior performance over the state-of-the-art.

In this paper, we develop a sparsity based model for the fusion of a high spatial-resolution image and a multispectral image. The given model is based on the combination of a dynamic gradient sparsity (DGS) and an anisotropic spectral-spatial total variation (ASSTV). We design an alternating direction method of multipliers (ADMM) based algorithm to solve the proposed model. In contrast to existing approaches, the proposed method can generate more spatial details as well as preserve favorable spectral information. Experimental results demonstrate that the proposed approach outperforms several state-of-the-art image fusion methods both quantitatively and visually, in terms of both pansharpening application of remote sensing images and fusion application of natural color images.

We study a truncated difference of $l_1$ and $l_2$ norms as a nonconvex metric for recovering sparse vectors and low-rank matrices from linear measurements. By discarding large magnitude entries/singular values in penalization, the proposed metric, denoted as truncated $l_{1-2}$, achieves a nearly unbiased approximation of the vector sparsity/matrix rank. We establish exact and stable recovery conditions of truncated $l_{1-2}$ minimization under the restricted isometry property (RIP) framework. Computationally, we apply the difference of convex functions algorithm (DCA) to efficiently solve truncated $l_{1-2}$ minimization with guaranteed convergence. Our method is validated on sparse vector recovery, matrix completion, and magnetic resonance imaging (MRI) reconstruction, with performance comparable to the state of the art. Particularly for MRI reconstruction, it succeeds in reconstructing the $256\times 256$ Shepp--Logan phantom image from merely 7 radial lines.

Multi-task feature learning aims to identify the shared features among tasks to improve generalization. Recent works have shown that the non-convex learning model often returns a better solution than the convex alternatives. Thus a non-convex model based on the capped-1, 1 regularization was proposed in [1], and the corresponding efficient multi-stage multi-task feature learning algorithm (MSMTFL) was presented. However, this method harnesses a fixed threshold in the capped-1, 1 regularization. The lack of adaptivity might result in suboptimal practical performance. In this paper we propose to employ an adaptive threshold in the capped-1, 1 regularized formulation, and the corresponding variant of MSMTFL will incorporate an additional scheme to adaptively determine the threshold. Considering that this threshold aims to distinguish true nonzero components of large magnitude from others, the heuristic of detecting the "first significant jump" proposed in [2] is applied here to adaptively determine its value. The preliminary theoretical analysis is provided to guarantee the feasibility of the proposed method. Several numerical experiments demonstrate the proposed method outperforms existing state-of-the-art feature learning approaches.

Image super-resolution (SR), a process to enhance image resolution, has important applications in satellite imaging, high-definition television, medical imaging, and so on. Many existing approaches use multiple low-resolution (LR) images to recover one high-resolution (HR) image. In this paper, we present an iterative scheme to solve single-image SR problems. It recovers a high-quality HR image from solely one LR image without using a training data set. We solve the problem from image intensity function estimation perspective and assume that the image contains smooth and edge components. We model the smooth components of an image using a thin-plate reproducing kernel Hilbert space and the edges using approximated Heaviside functions. The proposed method is applied to image patches, aiming to reduce computation and storage. Visual and quantitative comparisons with some competitive approaches show the effectiveness of the proposed method.

In this paper, we propose an automatic and efficient method to enhance edge sharpness of images. Starting from an image with blur edges, we improve the edges using transformed Heaviside functions for better visualization. In addition, we provide an efficient method to directly compute the scaling and shifting factors of the transformed Heaviside functions, so that blur edges can be improved accurately. Experimental results show that the proposed method is fast and can get sharper image edges than some recent state-of-the-art edge enhancement methods. We also apply the edge sharpening method to image super-resolution and obtained promising results.

The main aim of this paper is to study total variation (TV) regularization in deblurring and sparse unmixing of hyperspectral images. In the model, we also incorporate blurring operators for dealing with blurring effects, particularly blurring operators for hyperspectral imaging whose point spread functions are generally system dependent and formed from axial optical aberrations in the acquisition system. An alternating direction method is developed to solve the resulting optimization problem efficiently. According to the structure of the TV regularization and sparse unmixing in the model, the convergence of the alternating direction method can be guaranteed. Experimental results are reported to demonstrate the effectiveness of the TV and sparsity model and the efficiency of the proposed numerical scheme, and the method is compared to the recent Sparse Unmixing via variable Splitting Augmented Lagrangian and TV method by Iordache

As Google claims on its webpage, PageRank™ is the heart of software and continues to provide the basis for all of web search tools. In this algorithm, one page's PageRank value is divided evenly among all its outlinks. This paper discusses the value not-even distributed question, and puts forward an improved PageRank algorithm. By illustrating examples, we verify the effectiveness of our new algorithm and especially immunity to electronic spam.