Leveraging intrinsic data priors is critical for effective data recovery. However, existing approaches often struggle to achieve theoretical guarantees, strong performance, and computational efficiency simultaneously. In this paper, we introduce a novel Representative Coefficient Correlated Total Variation (RCCTV) regularizer that captures the recently observed low-rank and local smoothness properties of the representative coefficient tensor derived from a low-rank decomposition. RCCTV regularizer offers three key advantages: (1) it operates on a compact representative coefficient image significantly smaller than the original data, enabling highly efficient optimization; (2) it jointly enforces low-rankness and spatial smoothness through a single regularizer, eliminating the need for trade-off parameters; and (3) when integrated into a robust PCA framework (i.e., RCCTV-RPCA model), it admits provable exact recovery under mild conditions. To solve the resulting model, we develop an efficient ADMM-based algorithm accelerated via fast Fourier transform. Extensive experiments on both synthetic and real-world datasets demonstrate that the RCCTV-RPCA model achieves state-of-the-art accuracy while running significantly faster. Our code and Supplementary Material are available at https://github.com/mendy-2013/RCCTV.

Label Distribution Learning (LDL) is a novel machine learning paradigm that addresses the problem of label ambiguity and has found widespread applications. However, obtaining complete label distributions in real-world scenarios is challenging, which has led to the emergence of Incomplete Label Distribution Learning (InLDL). Existing InLDL methods attempt to utilize low-rank label correlations to recover the complete label distribution. However, we find that real-world LDL datasets have animbalancednature; that is, the sum of the description degrees for normal labels is significantly larger than that for tail labels, which disrupts the low-rank assumption underlying the recovery of the label distribution. To solve the above problem, we propose Incomplete and Imbalance Label Distribution Learning (I2LDL), which makes the use of low-rank label correlations more reasonable for InLDL. Our method decomposes the recovered label distribution matrix into a low-rank component for frequent labels and a sparse component for tail labels, effectively capturing the structure of both head and tail labels. We further require that the entries in the observed positions of the recovered label distribution matrix be close to the observed values, and that the recovered label distribution for every instance forms a probability simplex (i.e., nonnegative entries summing to unity). Finally, the proposed model is optimized via the Alternating Direction Method of Multipliers (ADMM). We provide a theoretical analysis of its exact recovery guarantee under standard assumptions of incoherence, sparsity, and sufficient sampling. Furthermore, we establish a generalization error bound based on Rademacher complexity, offering theoretical insights into the learning performance of our method. Extensive experiments on 16 real-world datasets demonstrate the effectiveness and robustness of our framework compared to existing InLDL methods. The code is available at https://anonymous.4open.science/r/IncomLDL-tailaware-C021.

Multi-label learning (MLL) has gained attention for its ability to represent real-world data. Label Distribution Learning (LDL), an extension of MLL to learning from label distributions, faces challenges in collecting accurate label distributions. To address the issue of biased annotations, based on the low-rank assumption, existing works recover true distributions from biased observations by exploring the label correlations. However, recent evidence shows that the label distribution tends to be full-rank, and naive apply of low-rank approximation on biased observation leads to inaccurate recovery and performance degradation. In this paper, we address the LDL with biased annotations problem from a novel perspective, where we first degenerate the soft label distribution into a hard multi-hot label and then recover the true label information for each instance. This idea stems from an insight that assigning hard multi-hot labels is often easier than assigning a soft label distribution, and it shows stronger immunity to noise disturbances, leading to smaller label bias. Moreover, assuming that the multi-label space for predicting label distributions is low-rank offers a more reasonable approach to capturing label correlations. Theoretical analysis and experiments confirm the effectiveness and robustness of our method on real-world datasets.

Real-world datasets like multi-spectral images and videos are naturally represented as tensors. However, limitations in data acquisition often lead to corrupted or incomplete tensor data, making tensor recovery a critical challenge. Solving this problem requires exploiting inherent structural patterns, with the low-rank property being particularly vital. An important category of existing low-rank tensor recovery methods relies on the tensor nuclear norms. However, these methods struggle with either computational inefficiency or weak theoretical guarantees for large-scale data. To address these issues, we propose a fast guaranteed tensor recovery framework based on a new tensor nuclear norm. Our approach adaptively extracts a column-orthogonal matrix from the data, reducing a large-scale tensor into a smaller subspace for efficient processing. This dimensionality reduction enhances speed without compromising accuracy. The recovery theories of two typical models are established by introducing an adjusted incoherence condition. Extensive experiments demonstrate the effectiveness of the proposed method, showing improved accuracy and speed over existing approaches. Our code and supplementary material are available at https://github.com/andrew-pengjj/adaptive_tensor_nuclear_norm.

Pseudo label based semi-supervised learning (SSL) for single-label and multi-label classification tasks has been extensively studied; however, semi-supervised label distribution learning (SSLDL) remains a largely unexplored area. Existing SSL methods fail in SSLDL because the pseudo-labels they generate only ensure overall similarity to the ground truth but do not preserve the ranking relationships between true labels, as they rely solely on KL divergence as the loss function during training. These skewed pseudo-labels lead the model to learn incorrect semantic relationships, resulting in reduced performance accuracy. To address these issues, we propose a novel SSLDL method called \textit{RankMatch}. \textit{RankMatch} fully considers the ranking relationships between different labels during the training phase with labeled data to generate higher-quality pseudo-labels. Furthermore, our key observation is that a flexible utilization of pseudo-labels can enhance SSLDL performance. Specifically, focusing solely on the ranking relationships between labels while disregarding their margins helps prevent model overfitting. Theoretically, we prove that incorporating ranking correlations enhances SSLDL performance and establish generalization error bounds for \textit{RankMatch}. Finally, extensive real-world experiments validate its effectiveness.

In recent years, tensor decomposition-based approaches forhyperspectral anomaly detection(HAD) have gained significant attention in the field of remote sensing. However, existing methods often fail to flexibly and effectively extract both the global correlations and local smoothness of the background components inhyperspectral images(HSIs). To mitigate this critical issue, we put forward a novel HAD method named HAD-EUNTRFR, which incorporates an enhanced unified nonconvex tensor ring (TR) factors regularization. In the HAD-EUNTRFR framework, the raw HSIs are first decomposed into background and anomaly components using the idea of tensor robust principal component analysis. The TR decomposition is then employed to capture the spatial-spectral correlations within the background component. Additionally, we introduce a unified and efficient nonconvex regularizer, induced bytensor singular value decomposition(T-SVD), to simultaneously encode the low-rankness and sparsity of the 3-D gradient TR factors into a unique concise form. The above characterization scheme enables the interpretable gradient TR factors to inherit the low-rankness and smoothness of the original background. To further enhance anomaly detection, we design a generalized nonconvex regularization term to exploit the group sparsity of the anomaly component. Based upon the above, we ultimately propose a scalable and reliable nonconvex HAD model. To solve the resulting doubly nonconvex model, we develop a highly efficient optimization algorithm based on thealternating direction method of multipliers(ADMM) framework. Theoretical results on convergence analysis for the proposed algorithm are derived. Experimental results on several benchmark datasets demonstrate that our proposed method outperforms existingstate-of-the-art(SOTA) approaches in terms of detection accuracy.

Abstract. Recently, the CTV-RPCA model proposed the first recoverable theory for separating low-rank and local-smooth matrices and sparse matrices based on the correlated total variation (CTV) regularizer. However, the CTV-RPCA model ignores the influence of noise, which makes the model unable to effectively extract low-rank and local-smooth principal components under noisy circumstances. To alleviate this issue, this article extends the CTV-RPCA model by considering the influence of noise and proposes two robust models with parameter adaptive adjustment, i.e., Stable Principal Component Pursuit based on CTV (CTV-SPCP) and Square Root Principal Component Pursuit based on CTV (CTV-[Formula: see text]). Furthermore, we present a statistical recoverable error bound for the proposed models, which allows us to know the relationship between the solution of the proposed models and the ground-truth. It is worth mentioning that, in the absence of noise, our theory degenerates back to the exact recoverable theory of the CTV-RPCA model. Finally, we develop the effective algorithms with the strict convergence guarantees. Extensive experiments adequately validate the theoretical assertions and also demonstrate the superiority of the proposed models over many state-of-the-art methods on various typical applications, including video foreground extraction, multispectral image denoising, and hyperspectral image denoising. The source code is released at https://github.com/andrew-pengjj/CTV-SPCP .

Infrared small target detection (ISTD) is a challenging task in the computer vision field due to factors such as target scale variations and strong clutter. The existing infrared patch tensor (IPT) models achieve good detection performance but still have several limitations, such as inaccurate background modeling results and poor robustness against noise. To alleviate these issues, in this article, we propose a new IPT model (dubbed as IPT-TCTV) by fully exploiting prior background knowledge. We construct an improved spatial-temporal (STT) model by sliding a 3-D window, which could better preserve the spatial correlation and temporal continuity of multiframe infrared images in the constructed tensor. Specifically, a joint low-rank and local smoothness regularization, i.e., tensor correlated total variation (TCTV), is utilized to characterize the background since the background exhibits not only the low-rank property but also the local smoothness property, without introducing additional trade-off parameters. Furthermore, considering the effect of edge structures, the${l} _{2,1}$norm is adopted as a noise constraint to eliminate strong residuals, which can help to extract real targets from the background with more precision. Finally, we design an efficient alternating direction method of multipliers (ADMMs) approach to solve the proposed model. Experimental results on some benchmark datasets illustrate that our IPT-TCTV model can achieve better detection performance than other state-of-the-art (SOTA) methods in various real scenes. The source code is released athttps://github.com/AuroraPei/IPT-TCTV.

Fully characterizing the spatial-spectral priors of hyperspectral images (HSI) is crucial for HSI denoising tasks. Recently, HSI denoising models based on representative coefficient images (RCIs) under the spectral low-rank decomposition framework have garnered significant attention due to their clever utilization of spatial-spectral information in HSI at a low cost. However, current methods either employ handcrafted classical denoisers or off-the-shelf deep denoisers to denoise RCIs, failing to fully capture the structural information of RCIs. In this paper, we propose a specific optimization framework for learning an RCI denoiser under the low-rank decomposition framework for the first time. Since low-rank decomposition can characterize the global low-rank property of HSI, our RCI denoiser only needs to learn the spatial prior of RCIs. Consequently, our optimization framework is inclined to learn a more powerful RCI denoiser. However, learning an RCI denoiser is not an easy task, primarily due to the lack of paired clean-noisy RCI data. To address this issue, we employ parametric techniques to represent the to-be-restored HSI as a function of RCI denoiser network parameters. In this way, the parameters of the RCI denoiser can thus be updated using noisy-clean HSI pairs. Furthermore, we adopt residual learning and Gaussian whitening techniques to enhance the RCI denoiser’s denoising ability for HSIs with various noise levels and different rank settings. Extensive experiments demonstrate that our method can achieve significant improvements in both denoising effectiveness and speed compared to state-of-the-art methods. The code of our algorithm is released at https://github.com/andrew-pengjj/RCILD.git.

Anomaly detection in hyperspectral images (HSIs) aims to identify sparse, interesting anomalies against the background, which has become a significant topic in remote sensing. Although the existing tensor-based methods have achieved commendable performance to some extent, there is still room for further improvement. In combination with three key techniques, i.e., gradient map-based modeling, circular tensor ring (TR) unfolding, and nonconvex regularization, this article proposes a novel generalized nonconvex method for hyperspectral anomaly detection (HAD) tasks within the TR framework. For the implementation of our proposed approach, abbreviated as TR-GNHAD, we first develop an effective and reliable HAD model in virtue of two newly unified nonconvex regularizers. The first regularizer is devised under a new prior characterization paradigm, which has a strong ability to encode two insightful prior information underlying the HSI’s background simultaneously, i.e., global low rankness and local smoothness. The other regularizer can well capture the structured sparsity of the abnormal component. Then, we derive an efficient optimization algorithm to solve the proposed model based on the alternating direction method of multipliers (ADMMs) framework. Experiments conducted on 12 HSI datasets illustrate that the proposed approach achieves highly competitive performance in both qualitative and quantitative metrics compared with several state-of-the-art HAD methods.

Within the tensor singular value decomposition (T-SVD) framework, existing robust low-rank tensor completion approaches have made great achievements in various areas of science and engineering. Nevertheless, these methods involve the T-SVD based low-rank approximation, which suffers from high computational costs when dealing with large-scale tensor data. Moreover, most of them are only applicable to third-order tensors. Against these issues, in this article, two efficient low-rank tensor approximation approaches fusing random projection techniques are first devised under the order-d ( d ≥ 3 ) T-SVD framework. Theoretical results on error bounds for the proposed randomized algorithms are provided. On this basis, we then further investigate the robust high-order tensor completion problem, in which a double nonconvex model along with its corresponding fast optimization algorithms with convergence guarantees are developed. Experimental results on large-scale synthetic and real tensor data illustrate that the proposed method outperforms other state-of-the-art approaches in terms of both computational efficiency and estimated precision.

In recent years, the imputation of spatiotemporal traffic data has emerged as a critical area of research within intelligent transportation systems. A commonly employed approach is low-rank matrix/tensor completion combined with additional spatiotemporal regularization techniques. However, many existing methods simply incorporate these regularizations into the low-rank model, resulting in models that are highly sensitive to trade-off parameters and yield mediocre results. Motivated by this problem, in this paper, we propose a method called Low-Rank Tensor Completion with 3D Spatiotemporal Transform (LRTC-3DST) for traffic data imputation, which effectively integrates low-rankness with three types of spatiotemporal characteristics in a fused manner and thus avoids trade-off parameters. We first design three specific transformations including graph Laplacian transform, fractional difference transform and periodic circulant transform for encoding the spatial local consistency, temporal local consistency and approximate periodicity of 3D traffic data, respectively. Then the LRTC-3DST model is proposed by using the truncated tensor nuclear norm on the three spatiotemporal feature tensors. The proposed model is optimized via the alternating direction multipliers method. Extensive experiments on a series of real traffic datasets demonstrate that our LRTC-3DST significantly outperforms numerous related methods, even when the missing rate is as high as 99%. The code is available athttps://github.com/HaoShu2000/LRTC-3DST.

A plethora of previous studies indicates that making full use of multifarious intrinsic properties of primordial data is a valid pathway to recover original images from their degraded observations. Typically, both low-rankness and local-smoothness broadly exist in real-world tensor data such as hyperspectral images and videos. Modeling based on both properties has received a great deal of attention, whereas most studies concentrate on experimental performance, and theoretical investigations are still lacking. In this paper, we study the tensor compressive sensing problem based on the tensor correlated total variation, which is a new regularizer used to simultaneously capture both properties existing in the same dataset. The new regularizer has the outstanding advantage of not using a trade-off parameter to balance the two properties. The obtained theories provide a robust recovery guarantee, where the error bound shows that our model certainly benefits from both properties in ground-truth data adaptively. Moreover, based on the ADMM update procedure, we design an algorithm with a global convergence guarantee to solve this model. At last, we carry out experiments to apply our model to hyperspectral image and video restoration problems. The experimental results show that our method is prominently better than many other competing ones. Our code and Supplementary Material are available at https://github.com/fsliuxl/cs-tctv.

Low-rank matrix factorization (LRMF) is a canonical problem in non-convex optimization, the objective function to be minimized is non-convex and even non-smooth, which makes the global convergence guarantee of gradient-based algorithm quite challenging. Recent work made a breakthrough on proving that standard gradient descent converges to the $\varepsilon$-global minima after $O( \frac{d \kappa^2}{\tau^2} {\rm ln} \frac{d \sigma_d}{\tau} + \frac{d \kappa^2}{\tau^2} {\rm ln} \frac{\sigma_d}{\varepsilon})$ iterations from small initialization with a very small learning rate (both are related to the small constant $\tau$). While the dependence of the convergence on the \textit{condition number} $\kappa$ and \textit{small learning rate} makes it not practical especially for ill-conditioned LRMF problem. In this paper, we show that precondition helps in accelerating the convergence and prove that the scaled gradient descent (ScaledGD) and its variant, alternating scaled gradient descent (AltScaledGD) converge to an $\varepsilon$-global minima after $O( {\rm ln} \frac{d}{\delta} + {\rm ln} \frac{d}{\varepsilon})$ iterations from general random initialization. Meanwhile, for small initialization as in gradient descent, both ScaledGD and AltScaledGD converge to $\varepsilon$-global minima after only $O({\rm ln} \frac{d}{\varepsilon})$ iterations. Furthermore, we prove that as a proximity to the alternating minimization, AltScaledGD converges faster than ScaledGD, its global convergence does not rely on small learning rate and small initialization, which certificates the advantages of AltScaledGD in LRMF.

Tensor recovery is a fundamental problem in tensor research field. It generally requires to explore intrinsic prior structures underlying tensor data, and formulate them as certain forms of regularization terms for guiding a sound estimate of the restored tensor. Recent researches have made significant progress by adopting two insightful tensor priors, i.e., global low-rankness (L) and local smoothness (S), which are always encoded as a sum of two separate regularizers into recovery models. However, unlike the primary theoretical developments on low-rank tensor recovery, these joint "L+S" models have no theoretical exact-recovery guarantees yet, making the methods lack reliability in real practice. To this crucial issue, in this work, we build a unique regularizer termed as tensor correlated total variation (t-CTV), which essentially encodes both L and S priors of a tensor simultaneously. Especially, by equipping t-CTV into the recovery models, we can rigorously prove the exact recovery guarantees for two typical tensor recovery tasks, i.e., tensor completion and tensor robust principal component analysis. To the best of our knowledge, this should be the first exact-recovery results among all related "L+S" methods for tensor recovery. We further propose ADMM algorithms with fine convergence to solve the proposed models. Significant recovery accuracy improvements are observed in extensive experiments. Typically, our method achieves a workable performance when the missing rate is extremely large, e.g., 99.5%, for the color image inpainting task, while all its peers totally fail in such a challenging case. Code is released at https://github.com/wanghailin97.

The latest tensor recovery methods based on tensor Singular Value Decomposition (t-SVD) mainly utilize the tensor nuclear norm (TNN) as a convex surrogate of the rank function. However, TNN minimization treats each rank component equally and tends to over-shrink the dominant ones, thereby usually leading to biased solutions. To handle this critical issue, we put forward a weighted tensor Schantten-p (0q(0 < q ≤ 1) sparse regularization item on the extensively existed noises/outliers is incorporated into the WSTN minimization to enhance its robustness in the impulsive scenarios. Finally, we propose an efficient and scalable robust high-order tensor recovery method solving a double nonconvex optimization with convergence guarantees. Synthetic and real experiments demonstrate that the proposed approach outperforms the state-of-the-art ones in terms of both accuracy and computational complexity.

Mining structural priors in data is a widely recognized technique for hyperspectral image (HSI) denoising tasks, whose typical ways include model-based methods and data-based methods. The model-based methods have good generalization ability, while the runtime can hardly meet the fast processing requirements of the practical situations due to the large size of an HSI${\mathbf {X}}\in \mathbb {R}^{\textrm {MN}\times B}$. For the data-based methods, they perform relatively fast on new test data once they have been trained. However, their generalization ability is always insufficient. In this article, we propose a fast model-based approach via a novel regularizer named the representative coefficient total variation (RCTV) to simultaneously characterize the low-rank and local smooth properties. The RCTV regularizer is proposed based on the observation that the representative coefficient matrix${\mathbf {U}}\in \mathbb {R}^{\textrm {MN}\times R} (R\ll B)$obtained by orthogonally transforming the original HSI${\mathbf {X}}$can inherit the strong local-smooth prior of${\mathbf {X}}$. Since$R/B$is very small, the model based on the RCTV regularizer has lower time complexity. In addition, we find that the representative coefficient matrix${\mathbf {U}}$is robust to noise, and thus, the RCTV regularizer can somewhat promote the robustness of the HSI denoising model. Extensive experiments on mixed noise removal demonstrate that the proposed method realizes a perfect compromise between denoising performance and denoising speed compared with other state-of-the-art methods. Remarkably, the denoising speed of our proposed method outperforms all competing model-based techniques and is comparable with the deep learning-based approaches. The code of our algorithm is released athttps://github.com/andrew-pengjj/rctv.git.

Recently, tensor Singular Value Decomposition (t-SVD)-based low-rank tensor completion (LRTC) has achieved unprecedented success in addressing various pattern analysis issues. However, existing studies mostly focus on third-order tensors while order- d ( d ≥ 4 ) tensors are commonly encountered in real-world applications, like fourth-order color videos, fourth-order hyper-spectral videos, fifth-order light-field images, and sixth-order bidirectional texture functions. Aiming at addressing this critical issue, this paper establishes an order- d tensor recovery framework including the model, algorithm and theories by innovatively developing a novel algebraic foundation for order- d t-SVD, thereby achieving exact completion for any order- d low t-SVD rank tensors with missing values with an overwhelming probability. Emperical studies on synthetic data and real-world visual data illustrate that compared with other state-of-the-art recovery frameworks, the proposed one achieves highly competitive performance in terms of both qualitative and quantitative metrics. In particular, as the observed data density becomes low, i.e., about 10%, the proposed recovery framework is still significantly better than its peers. The code of our algorithm is released at https://github.com/Qinwenjinswu/TIP-Code.

The tensor-tensor product-induced tensor nuclear norm (t-TNN) (Lu et al., 2020) minimization for low-tubal-rank tensor recovery attracts broad attention recently. However, minimizing the t-TNN faces some drawbacks. For example, the obtained solution could be suboptimal to the original problem due to its loose approximation. In this article, we extract a unified nonconvex surrogate of the tensor tubal rank as a tighter regularizer, which involves many popular nonconvex penalty functions. An iterative reweighted t-TNN algorithm is proposed to solve the resulting generalized nonconvex tubal rank minimization for tensor recovery. It converges to a critical point globally with rigorous proofs based on the Kurdyka-Łojasiwicz property. Furthermore, we provide the theoretical guarantees for exact and robust recovery by developing the tensor null space property. Extensive experiments demonstrate that our approach markedly enhances recovery performance compared with several state-of-the-art convex and nonconvex methods.

In this paper, the generative model is used to introduce the structural properties of the signal to replace the common sparse hypothesis, and a non-convex compressed sensing sparse deviation model based on the generative model (ℓq-Gen) is proposed. By establishing ℓqvariant of the restricted isometry property (q-RIP) and Set-Restricted Eigenvalue Condition (q-S-REC), the error upper bound of the optimal decoder is derived when the recovered signal is within the sparse deviation range of the generator. Furthermore, it is proved that the Gaussian matrix satisfying a certain number of measurements is sufficient to ensure a good recovery for the generating function with high probability. Finally, a series of experiments are carried out to verify the effectiveness and superiority of the ℓq-Gen model.

The recently proposed Tensor Nuclear Norm (TNN) minimization has been widely used for tensor completion. However, previous works didn’t consider the structural difference between the observed data and missing data, which widely exists in many applications. In this paper, we propose to incorporate a constraint item on the missing values into low-tubal-rank tensor completion to promote the structural hypothesis of the missing values such as sparsity. Theoretically, the proposed model has lower recovery error than classical model, and the target tensor can be recovered exactly with overwhelming probability provided low-tubal-rankness on whole area and sparsity on missing area. Algorithmically, an efficient algorithm by Alternating Direction Method of Multiplier (ADMM) is presented. Extensive experiments on both synthetic and real-world data demonstrate its superiority compared with several state-of-the-art methods.

Recently, deep learning methods have made a remarkable improvement in compressed sensing image recovery stage. In the compressed measurement stage, the existing methods measured by block by block owing to a huge measurement dictionary for the whole images and the high computational complexity. In this work, a novel deep convolutional neural network (DCNN) named Convolutional Measurement Compressed Sensing network (CMCS‐net) is proposed for image compressed sensing considering both convolutional measurement (CM) and sparse prior. Different from existing works, the convolution operation is adopted both in the measurement phase and reconstruction phase, which retains the structure information of images much better. Simultaneously, the size of the measurement matrix is no longer limited by data dimensions. Particularly, by unfolding the CM process to analyse a Toeplitz‐type matrix, the theoretical support of the convolutional compressed measurement is proposed. In addition, in the recovery phase, the authors consider the sparse prior in nature images by embedding the truncated hierarchical projection algorithm into their architecture to solve the problem of multilayered convolutional sparse coding. Furthermore, extensive experiments demonstrate that their proposed CMCS‐net can marvellously reconstruct the images and fully remove the block artefact.