Tensor decompositions are powerful tools for capturing the low-rank structure of dynamic videos. However, existing tensor decompositions primarily consider pixel-wise interactions, thus capturing solely global spatio-temporal correlations and struggling to handle the complex patterns that are inherent to dynamic videos in real-world applications. To overcome this limitation, we propose a dynamic Bhattacharya-Mesner (DyBM) decomposition, which represents the dynamic video as a sum of terms, with each term being the convolution of a BM-rank 1 tensor and a learnable three-dimensional filter. The newly constructed filters enable DyBM decomposition to establish patch-wise interactions in BM-rank 1 tensors, effectively capturing both global and local spatio-temporal correlations in dynamic videos. We further provide a physical interpretation of the factors in DyBM decomposition and offer an in-depth discussion of its relationship to the original BM decomposition. To evaluate the effectiveness of DyBM decomposition, we build a dynamic video recovery model. To solve the model, we develop a corresponding optimization algorithm with a theoretical convergence guarantee. Extensive experiments verify that DyBM decomposition-based method performs more favorably than the state-of-the-art tensor decomposition-based methods especially for dynamic videos.

Thick cloud removal is a long-standing and critical challenge in remote sensing (RS) image processing, with the increasing availability of multi-temporal RS images offering new opportunities to address this problem. The main limitation of existing cloud removal methods is that the classical fidelity only considers original pixel domain or handcrafted/pretrained filtered domains, overlooking the individuality filters and the corresponding feature behind each RS image, which leads to evident detail discrepancies. To address this issue, we suggest a data-aware fidelity based on the untrained neural network, which encourages deep data-aware feature matching between the contaminated image and the guidance image. Complementary to the data-aware fidelity, we design the deep self-representation to implicitly impose regularization benefiting from the same untrained neural network. Equipped with the elaborately designed fidelity and regularization, we propose a unified data-aware fidelity and regularization learning (called DAFRL) paradigm for thick cloud removal that flexibly adapts to diverse multi-temporal RS images. Under this paradigm, the fidelity and regularization are empowered by the same untrained neural network, serving distinct functions while collaborating organically. Experimental results on both simulated and real datasets show that the proposed DAFRL effectively preserves fine details and outperforms the compared methods.

Hyperspectral anomaly detection is an important task in remote sensing for identifying pixels with anomalous spectral signatures that deviate from their local background. Recently, low-rank and sparse representation-based methods have garnered significant attention in hyperspectral anomaly detection, which typically employ low-rank representation to characterize the background and sparse representation to capture anomalies. Since the background and anomalies usually exhibit complex characteristics beyond the low-rankness and sparsity, low-rank and sparse representation-based methods typically do not perform satisfactorily for complex scenarios. To address the challenge, we propose an unsupervised hyperspectral anomaly detection method from a continuous perspective, which organically integrates Continuous Background representation and deep Anomaly Representation (CBAR). Specifically, the CBAR model leverages the continuous low-rank tensor function to encapsulate both the low-rankness and smoothness of the background and the deep neural network to capture the complex geometric structure of anomalies. Moreover, to mitigate the overfitting of the background and anomalies to the observed HSI, we introduce two terms as overfit-shield by exploiting the prior knowledge of the background and anomalies. To solve the CBAR model, we develop an efficient alternating minimization algorithm. Extensive experiments on benchmark datasets (including Airport, Urban, Beach, and HYDICE) demonstrate that the proposed CBAR outperforms the state-of-the-art anomaly detection methods both qualitatively and quantitatively. For reproducibility, we will release our source code at: https://github.com/Weihao-Wu/CBAR.

Recently, the tensor nuclear norm (TNN)-based tensor robust principle component analysis (TRPCA) has achieved impressive performance in multidimensional data processing. The underlying assumption in TNN is the low-rankness of frontal slices of the tensor in the transformed domain (e.g., Fourier domain). However, the low-rankness assumption is usually violative for real-world multidimensional data (e.g., video and image) due to their intrinsically nonlinear structure. How to effectively and efficiently exploit the intrinsic structure of multidimensional data remains a challenge. In this article, we first suggest a kernelized TNN (KTNN) by leveraging the nonlinear kernel mapping in the transform domain, which faithfully captures the intrinsic structure (i.e., implicit low-rankness) of multidimensional data and is computed at a lower cost by introducing kernel trick. Armed with KTNN, we propose a tensor robust kernel PCA (TRKPCA) model for handling multidimensional data, which decomposes the observed tensor into an implicit low-rank component and a sparse component. To tackle the nonlinear and nonconvex model, we develop an efficient alternating direction method of multipliers (ADMM)-based algorithm. Extensive experiments on real-world applications collectively verify that TRKPCA achieves superiority over the state-of-the-art RPCA methods.

The transform-based tensor nuclear norm (TNN) methods have shown promise in hyperspectral images (HSIs) mixed noise removal. However, these transform-based methods only consider the low-rankness of the transformed tensor. The sparsity of the transformed tensor, which is beneficial for HSIs denoising, is neglected in the transform-based TNN methods. In this paper, we propose a novel HSIs denoising model based on the group-tube transform induced collaborative sparsity and low-rankness (GTCSLR). Here, GTC-SLR organically integrates the sparsity and low-rankness, capitalizing on their synergies to enhance denoising performance. We develop a proximal alternating minimization (PAM)-based algorithm to efficiently address the resulting optimization problem. Experimental results demonstrate the superiority of our method over state-of-the-art HSIs denoising methods, as evidenced by quality metrics and visual assessments.

In the field of remote sensing (RS) imaging, the occurrence of adverse meteorological conditions or sensor malfunctions can lead to missing data, posing a substantial impediment. Low-rank tensor decomposition has emerged as a promising strategy for resolving this issue, as it enables the integration of diverse data priors within a unified framework. Although various decomposition techniques, such as Tucker decomposition and tensor ring decomposition (TRD), have been developed based on multilinear products, they may not adequately capture the complex structure of RS imagery. Therefore, there is a need for tensor decompositions that incorporate nonlinear operations. To alleviate this challenge, a multi-nonlinear product is defined, which enables the construction of a nonlinear Tucker decomposition (NTD) model. To enhance the model’s capability, a stacked Tucker decomposition (STD) model is formulated, by representing a tensor as the product of a core tensor and a collection of factor matrices along each mode, utilizing the multi-nonlinear product, which potentially regulates the distribution of singular values, thereby achieving a more accurate characterization of textures. The proposed model, integrated with total variation regularization, is subsequently applied to the task of RS imagery inpainting. Extensive experimental results demonstrate the superiority of the proposed model over state-of-the-art (SOTA) methods across various tasks. This validates its effectiveness and adaptability in mitigating the challenges associated with RS imagery inpainting. The code is available athttps://github.com/shuangxu96/STDTV.

Abstract. The transform-based tensor nuclear norm (TNN) methods have shown good recovery results for tensor completion. However, the TNN methods are based on the single-tube transforms in which transforms are applied to each tube independently. The performance of the single-tube transform-based TNN methods is not good for recovery of missing tubes in multidimensional images (e.g., all the observations are missing in a pixel location of multispectral images). The main aim of this paper is to address this issue by proposing and developing a learnable group-tube transform-based TNN (GTNN) method that can effectively explore the correlation of neighboring tubes by leveraging a learnable group-tube transform. The proposed learnable group-tube transform is a separable three-dimensional transform that consists of a one-dimensional spectral/temporal transform (i.e., single-tube transform) and a two-dimensional spatial transform. Such group-tube transform can effectively explore the correlation of neighboring tubes. Based on the elaborately designed low-rank metric GTNN, we suggest a low-rank tensor completion model. To solve this highly nonconvex model, we design an efficient multiblock proximal alternating minimization algorithm and establish the convergence guarantee. A variety of numerical experiments on real-world multidimensional imaging data including traffic speed data, color images, videos, and multispectral images collectively manifest that the GTNN method outperforms some state-of-the-art TNN methods especially when the observations along tubes are missing.

Through the development of Synthetic Aperture Radar (SAR) technology, it is now possible to observe dynamic processes on the earth with fine temporal resolution by forming SAR time series. Nonetheless, such sequential images remain difficult to interpret due to the speckle effect. Despeckling them is further complicated by outliers caused by abrupt changes in weather conditions or the appearance of objects. In spite of the fact that many state-of-the-art methods can achieve excellent filtering performances over stable areas, they often result in artifacts in those areas where outliers existed at the time of acquisition. To simultaneously mitigate the speckle noise and extract outliers, we propose a novel SAR time series despeckling method based on nonlocal total variation regularized robust principle component analysis, which is termed SAR-NL-TVRPCA. By comparing it to other state-of-the-art methods, the effectiveness of its despeckling has been validated in real data experiments. Furthermore, the extracted outliers can provide insight into abrupt changes occurring throughout the observation period, which provides byproducts for further analysis.

The existing low-rank tensor completion methods develop many tensor decompositions and corresponding tensor ranks in order to reconstruct the missing information by exploiting the inherent low-rank structure under the assumption that the data is low-rank under one of the kinds of decompositions. However, the assumption is easily violated for real-world data, e.g., color images and multispectral images, as the low-rank structure of these data is not significant. To better take advantage of the global correlation relationship, we propose a kernel low-rank tensor completion model, where original data is mapped into the feature space using a kernel mapping. Although the original data is high-rank, it is low-rank in the feature space owing to the kernel mapping. Therefore, the proposed model could take advantage of the implicitly low-rank structure in the feature space and estimate the missing entries well. Considering it is not easy to explicitly kernelize the tensor, we reformulate the model as the inner product form and introduce the kernel trick for efficiently solving the resulting model. Extensive experiments on color images and multispectral images show that the proposed method outperforms the state-of-the-art low-rank tensor completion methods.

The existing cloud removal methods either need a cloud mask as prior knowledge or detect clouds before cloud removal processing, i.e., the detection and removal processes are separate. In this article, we propose a box-constrained (BC) smooth low-rank plus group sparse model to simultaneously detect and remove clouds, by formulating the degraded data as the summation of image and cloud components. For the cloud component, we propose a group sparse function along the spectral dimension. This is motivated by our observations that: 1) one tube is contaminated by clouds if any pixels of this tube are contaminated by clouds; 2) the positions of tubes, which are taken at different times, contaminated by clouds are different. For the image component, we propose to use a tensor rank based on the tensor singular value decomposition. The tensor rank characterizes the global property of the image component and could not keep the cloud-free information unchanged. To address the problem, we introduce a BC on the image component to force its cloud-free information to be equal to the corresponding values of observed data. Owing to the BC, the proposed model integrates the cloud detection and removal processes so that the two processes could promote each other and result in a promising result. To solve the proposed model, we develop an efficient algorithm that can generate the cloud mask, image component, and cloud component alternately. Extensive experiments on synthetic and real data show that the proposed method is competitive compared with the completion and other cloud removal methods.

Recently, unsupervised deep learning-based methods have shown an empirical success in hyperspectral images (HSIs) denoising, profiting from the strong representation ability of convolutional neural networks. However, these methods only can describe the local structure of the spatial dimension, which is restricted to very limited local receptive fields. To overcome this difficulty, a novel HSIs denoising model based on the deep image prior (DIP) framework is proposed by adding a tensor ring (TR) decomposition, which can enlarge the receptive field of the spatial dimension and capture global spectral correlation simultaneously. Unlike the previous DIP framework that directly minimizes the objective function, we develop an algorithm based on proximal alternating minimization to decouple the model into the classic DIP framework and TR cores least-squares problems, which are easy to solve. Experimental results verify that the proposed DIP- TR compares favorably with compared methods in terms of quality metrics and visual performance.

The tensor tubal rank, defined based on the tensor singular value decomposition (t-SVD), has obtained promising results in hyperspectral image (HSI) denoising. However, the framework of the t-SVD lacks flexibility for handling different correlations along different modes of HSIs, leading to suboptimal denoising performance. This article mainly makes three contributions. First, we introduce a new tensor rank named tensor fibered rank by generalizing the t-SVD to the mode-k t-SVD, to achieve a more flexible and accurate HSI characterization. Since directly minimizing the fibered rank is NP-hard, we suggest a three-directional tensor nuclear norm (3DTNN) and a three-directional log-based tensor nuclear norm (3DLogTNN) as its convex and nonconvex relaxation to provide an efficient numerical solution, respectively. Second, we propose a fibered rank minimization model for HSI mixed noise removal, in which the underlying HSI is modeled as a low-fibered-rank component. Third, we develop an efficient alternating direction method of multipliers (ADMMs)-based algorithm to solve the proposed model, especially, each subproblem within ADMM is proven to have a closed-form solution, although 3DLogTNN is nonconvex. Extensive experimental results demonstrate that the proposed method has superior denoising performance, as compared with the state-of-the-art competing methods on low-rank matrix/tensor approximation and noise modeling.

Remotely sensed images may contain some missing areas because of poor weather conditions and sensor failure. Information of those areas may play an important role in the interpretation of multitemporal remotely sensed data. This paper aims at reconstructing the missing information by a nonlocal low-rank tensor completion method. First, nonlocal correlations in the spatial domain are taken into account by searching and grouping similar image patches in a large search window. Then, low rankness of the identified fourth-order tensor groups is promoted to consider their correlations in spatial, spectral, and temporal domains, while reconstructing the underlying patterns. Experimental results on simulated and real data demonstrate that the proposed method is effective both qualitatively and quantitatively. In addition, the proposed method is computationally efficient compared with other patch-based methods such as the recently proposed patch matching-based multitemporal group sparse representation method.