Snapshot compressive imaging (SCI) captures multispectral images (MSIs) using a single coded two-dimensional (2-D) measurement, but reconstructing high-fidelity MSIs from these compressed inputs remains a fundamentally ill-posed challenge. Recent diffusion-based methods improve quality but are limited by scarce MSI training data, domain shifts from RGB-pretrained models, and slow multi-step sampling. These drawbacks restrict their practicality in real-world applications. Unlike prior approaches that rely on expensive iterative refinement or subspace-based diffusion embeddings (e.g., DiffSCI, PSR-SCI)—we introduce a fundamentally different paradigm: a self-supervised One-Step Diffusion (OSD) framework designed specifically for SCI. The key novelty lies in using a single-step diffusion refiner to correct an initial reconstruction, eliminating iterative denoising entirely while preserving generative quality. Moreover, we adopt a self-supervised equivariant learning strategy to train both the predictor and refiner directly from raw 2-D measurements, enabling generalization to unseen domains without ground-truth MSI. To further address limited MSI data, we design a band-selection–driven distillation strategy that transfers core generative priors from large-scale RGB datasets, effectively bridging the domain gap. Extensive experiments confirm that our approach sets a new standard—yielding PSNR gains of 3.44dB, 1.61dB, and 0.28dB on the Harvard, NTIRE, and ICVL datasets respectively, while cutting reconstruction time from 8.9s to just 0.22s per image. These gains in efficiency and adaptability advance SCI reconstruction, enabling accurate and practical real-world deployment.

The presence of metallic implants introduces bright and dark streaks that appear in computed tomography (CT) images, degrading image quality and interfering with medical diagnosis. To reduce these artifacts, deep learning approaches have been applied for metal-corrupted restoration, which usually requires a large amount of simulated degraded-clean pairs for training. To achieve metal artifact reduction (MAR) without reference images, implicit neural representation (INR) has emerged and shown capabilities for image restoration in an unsupervised manner. However, existing INR methods for MAR usually treat the spatial coordinates independently and ignore their correlation, resulting in detail loss and artifacts remaining. In this paper, we propose an INR-based unsupervised MAR framework and design a High-order Line Attention Network to capture local contextual and geometric representations from X-rays, which maps the spatial coordinates into discrete linear attenuation coefficients of imaged objects for artifact-free CT image reconstruction. The second-order feature interaction can effectively improve the spectral bias problems and fit low and high-frequency details of real signals well. The proposed line-attention module with linear complexity can establish global relationships among spatial point tokens from sampled rays. To provide more local contextual information, a multiple local adjacent ray sampling strategy is adopted to compose several sub-fan beams with more context as a training batch. With the help of these components, the unsupervised MAR framework can approximate the implicit continuous function to estimate measurements and generate artifact-free CT images. Simulated and real experiments indicated that the proposed approach achieved superior MAR performance compared with other state-of-the-art methods.

Few-shot learning (FSL) has been employed in hyperspectral image (HSI) classification, achieving excellent performance with limited training data. However, existing HSI few-shot classification methods often encounter the problem of insufficient domain-transferable knowledge learning that is either from natural images or HSI solely. In this paper, we propose a two-stage cross-domain few-shot classification method of HSI, which for the first time makes use of heterogeneous labeled natural images and HSIs in the source domain (SD) to support the classification of novel classes in the target HSI domain. We first use a large amount of labeled natural images at the first stage to pre-train a backbone, which will be used to extract the spatial feature of HSIs at the second stage with fine-tuning. In the second stage, we propose a cross-domain few-shot classification method, which allows for effective discriminative feature learning in the target HSI domain with the transferred knowledge of the old classes obtained from natural images and HSIs in the source domain. To obtain domain-transferable knowledge, FSL is employed on the HSI source and target domain. To deal with the domain shift problem, we propose a class-matching based cross-domain contrastive loss. In addition, we take into account the large spectral variations problem in the target HSI domain and introduce an instance-level self-supervised loss. Experimental results on real data sets demonstrate that our method outperforms the recent state-of-the-art.

Recently, transformer-based models have achieved remarkable performance in the hyperspectral image (HSI) classification. However, due to the limited training data, existing methods often show limited capability of capturing fine-grained local features. Although attempts have been made to solve this problem, the large amount of parameters imposes the risk of overfitting. In this paper, we propose a locally enhanced transformer network for HSI classification with fewer network parameters, which mainly consists of a multi-branch spatial-spectral tokenization (MSST) module and a dual-branch transformer encoder (DTE) module. The MSST generates effective spatialspectral tokens through diverse convolutions with a residual connection. The DTE consists of a global transformer branch and a locally enhanced transformer branch, which are used to capture the global and local spatial dependencies of HSI, respectively. Unlike the conventional self-attention module used in the global branch, we propose an improved multi-head selfattention (IMSA) module in the local branch by incorporating the local prior information of HSI with graph convolution, to enhance the local information extraction. To fuse the global and local features from the two branches, we introduce an adaptive strategy by using learnable weights for both branches. We devise our MSST and DTE with a shallow architecture, significantly reducing the number of parameters. Experimental results on benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art.

Cross-domain few-shot learning (FSL) has shown promising performance in hyperspectral image classification (HSIC) under limited labeled data. However, existing approaches often suffer from insufficient meta-knowledge transfer due to reliance on a single source domain, and fail to fully bridge the domain gap owing to the use of single-level alignment strategies. In this paper, we propose a heterogeneous data-driven, global-to-local cross-domain FSL framework for HSIC, leveraging richly labeled natural RGB images and hyperspectral data as source domains to support classification in the target domain with only a few labeled samples. The proposed method consists of two stages. In the first stage, we employ natural RGB images to learn a powerful spatial feature extractor with FSL and self-supervised learning, which will be fine-tuned for HSI in the second stage. To alleviate the domain gap between the source and target HSI domains at the second cross-domain FSL stage, we propose a global-to-local domain adaptation strategy that performs alignment both at the domain level and class level, effectively reducing the learning bias toward the source HSI while promoting discriminative feature learning. Additionally, to address large intra-class variance in the target HSI domain, we introduce a self-supervised contrastive loss based on positive pairs only, enhancing the within-class representation compactness. Extensive experiments on three benchmark datasets demonstrate that our method outperforms the state-of-the-art.

Hyperspectral images (HSIs) play a pivotal role in fields, such as medical diagnosis and agriculture. However, it often contends with significant noise stemming from narrowband spectral filtering. Existing denoising techniques have their limitations: model-driven methods rely on manual priors and hyperparameters, while learning-based methods struggle to discern intrinsic noise patterns, as they require paired images with specific example noise for training, fail to capture critical noise distribution information, leading to unrobust denoising results. This work addresses the issue by presenting a degradation-noise-aware unfolding network (DNA-Net). Unlike training directly with the simulated noise, DNA-Net initially models general sparse and Gaussian noise through statistic distributions. It then explicitly represents image priors with a customized spectral transformer. The model is subsequently unfolded into an end-to-end (E2E) network, with hyperparameters adaptively estimated from noisy HSI and degradation models, effectively regulating each iteration. Furthermore, a novel U-shaped local-nonlocal–spectral transformer (U-LNSA) is introduced, simultaneously capturing spectral correlations, local features, and nonlocal dependencies. The integration of U-LNSA into DNA-Net establishes the first Transformer-based deep unfolding method for HSI denoising. Experimental results on synthetic and real noise validate DNA-Net’s superior performance over state-of-the-art (SOTA) methods. Moreover, the DNA-Net, trained exclusively on mixed Gaussian noise and impulse noise, demonstrates the ability to generalize to unseen noise present in real images. Code and models will be released at:https://github.com/NavyZeng/DNA-Net.

This paper introduces a novel approach to Generalized Category Discovery (GCD) by leveraging the concept of contextuality to enhance the identification and classification of categories in unlabeled datasets. Drawing inspiration from human cognition’s ability to recognize objects within their context, we propose a dual-context based method. Our model integrates two levels of contextuality: instance-level, where nearest-neighbor contexts are utilized for contrastive learning, and cluster-level, employing prototypical contrastive learning based on category prototypes. The integration of the contextual information effectively improves the feature learning and thereby the classification accuracy of all categories, which better deals with the real-world datasets. Different from the traditional semi-supervised and novel category discovery techniques, our model focuses on a more realistic and challenging scenario where both known and novel categories are present in the unlabeled data. Extensive experimental results on several benchmark data sets demonstrate that the proposed model outperforms the state-of-the-art. Code is availale at: https://github.com/Clarence-CV/Contexuality-GCD

Convolutional neural networks (CNN) have demonstrated excellent performance in the classification of hyperspectral image (HSI). However, CNN-based models often fail to capture long-range contextual information due to the limited receptive fields. In this paper, we propose a locally enhanced transformer network for HSI classification. Firstly, we propose a multi-branch spatial-spectral token (SST) module based on CNN to transform HSI into the spatial-spectral tokens, facilitating the reduction of information loss during tokenization. Secondly, after SST we propose a dual-branch transformer module, which consists of a global transformer and a locally enhanced transformer, to capture the global and local spatial features of HSI. Particularly, in the local branch we develop an improved multi-head self-attention (IMSA) by incorporating the neighbourhood information derived from super-pixel segmentation to improve the local feature extraction ability of the conventional transformer. Experimental results on benchmark datasets demonstrate that the proposed method achieves better performance over the state-of-the-art.

Subspace clustering has achieved remarkable performance for hyperspectral image (HSI). However, existing methods are often computationally expensive and have limited ability to capture the intrinsic structural information of HSI. In this paper, we propose a structural prior-guided subspace clustering method, which simultaneously incorporates the local and non-local spatial information and the cluster prior information. Accordingly, three efficient regularizations are developed. Considering the local connectivity of pixels, we propose an ℓ2,1norm based constraint on the representation difference matrix to improve the homogeneity of clustering result. Next, to capture the non-local geometric structure of HSI, we propose a manifold-based regularization with an adaptively learned landmark graph. Furthermore, we explore the block-diagonal cluster structure of HSI and develop a landmark-based clustering constraint, which makes the representations more favorable for clustering. Our local constraint is imposed on all the data points due to its efficiency and the latter two are solely imposed on landmarks, leading to computationally efficient regularizations. Due to the local constraint, the manifold and cluster structure of the landmarks can be effectively propagated to all the data points. To make our model scalable to large-scale data, we learn a compact dictionary with an orthogonal constraint, significantly reducing the number of parameters. In addition, we propose a novel landmark selection method to support our landmark-based constraints using multi-scale super-pixel segmentation and clustering, which improves the uniformity and diversity of landmarks. We also develop an efficient algorithm to solve the proposed model. Experimental results demonstrate that our model outperforms the state-of-the-art.

Recently, weak-to-strong consistency-based methods have yielded a remarkable performance for remote sensing image segmentation. However, they are designed within a single view, which encounters the problems of unreliable pseudo-label supervision and insufficient ability to capture informative features for the segmentation of complex remote sensing data. In this article, we propose a cross-view weak-to-strong consistency-based method, which aggregates rich information from two irrelevant views. We employ two subnets for the two views to generate view-specific features while encouraging them to yield the same prediction. Within each view, we enhance the perturbation space of data at the image level and feature level for a more robust representation. To leverage the information from unlabeled data, we propose a cross-view weak-to-strong consistency scheme, which employs the pseudo-label of the weakly augmented data in one view to supervise the model training in another view, facilitating an effective information exchange across views. To avoid identical information extraction from the two views, we propose a cross-view contrastive loss to maximize the dissimilarity of the feature representations across views, which ensures that the learned complementary information from one view provides additional helpful information for the model training in another view. Finally, we propose a cross-view discrepancy-based supervised constraint by imposing a larger weight on the areas that exist discrepant predictions across views in the cross-entropy loss, allowing the model to focus more on the hard-to-classify regions of the images. Extensive experimental results on several benchmark datasets demonstrate that our method outperforms the state-of-the-art.

In this article, we propose a novel bilayer low-rankness measure and two models based on it to recover a low-rank (LR) tensor. The global low rankness of underlying tensor is first encoded by LR matrix factorizations (MFs) to the all-mode matricizations, which can exploit multiorientational spectral low rankness. Presumably, the factor matrices of all-mode decomposition are LR, since local low-rankness property exists in within-mode correlation. In the decomposed subspace, to describe the refined local LR structures of factor/subspace, a new low-rankness insight of subspace: a double nuclear norm scheme is designed to explore the so-called second-layer low rankness. By simultaneously representing the bilayer low rankness of the all modes of the underlying tensor, the proposed methods aim to model multiorientational correlations for arbitrary N -way ( N ≥ 3 ) tensors. A block successive upper-bound minimization (BSUM) algorithm is designed to solve the optimization problem. Subsequence convergence of our algorithms can be established, and the iterates generated by our algorithms converge to the coordinatewise minimizers in some mild conditions. Experiments on several types of public datasets show that our algorithm can recover a variety of LR tensors from significantly fewer samples than its counterparts.

Hyperspectral image (HSI) captures rich spectral information in more than hundreds of spectral bands, which allows far better discrimination between ground objects compared with the conventional optical images. Therefore, HSIs find a number of applications in Earth observation, precision agriculture and environmental monitoring. However, due to the effect of poor imaging condition, hardware limitation and sensor noise, the acquisition of HSIs is inevitably affected by noise, which hinders accurate interpretation of HSI in real applications. A number of denoising methods have been proposed for HSI, including BM4D [1] , LRMR [2] , LRTV [3] . Recent works such as HSID-CNN [4] , which adopt deep learning based technique, have yielded the advanced performance. However, most of them design neural network architecture to process noisy data in a spatial domain. It is known that HSI has significantly different properties in high-frequency and low-frequency domains. For instance, in low-frequency domain, smoothing regions of HSI are more relevant while the details of HSI and noise are often more salient in the high-frequency domain. Current methods restore clean HSIs in a spatial domain, and thus neglect the properties of HSI in different frequency domains, which lead to limited denoising performance.

Band selection (BS) reduces effectively the spectral dimension of a hyperspectral image (HSI) by selecting relatively few representative bands, which allows efficient processing in subsequent tasks. Existing unsupervised BS methods based on subspace clustering are built on matrix-based models, where each band is reshaped as a vector. They encode the correlation of data only in the spectral mode (dimension) and neglect strong correlations between different modes, i.e., spatial modes and spectral mode. Another issue is that the subspace representation of bands is performed in the raw data space, where the dimension is often excessively high, resulting in a less efficient and less robust performance. To address these issues, in this article, we propose a tensor-based subspace clustering model for hyperspectral BS. Our model is developed on the well-known Tucker decomposition. The three factor matrices and a core tensor in our model encode jointly the multimode correlations of HSI, avoiding effectively to destroy the tensor structure and information loss. In addition, we propose well-motivated heterogeneous regularizations (HRs) on the factor matrices by taking into account the important local and global properties of HSI along three dimensions, which facilitates the learning of the intrinsic cluster structure of bands in the low-dimensional subspaces. Instead of learning the correlations of bands in the original domain, a common way for the matrix-based models, our model learns naturally the band correlations in a low-dimensional latent feature space, which is derived by the projections of two factor matrices associated with spatial dimensions, leading to a computationally efficient model. More importantly, the latent feature space is learned in a unified framework. We also develop an efficient algorithm to solve the resulting model. Experimental results on benchmark datasets demonstrate that our model yields improved performance compared to the state-of-the-art.

Clustering algorithms play an essential and unique role in classification tasks, especially when annotated data are unavailable or very scarce. Current clustering approaches in remote sensing are mostly designed for a single data source, such as hyperspectral image (HSI), while, nowadays, multisensor data are being routinely acquired. In this article, we propose a multiview subspace clustering model that exploits effectively the rich information from multiple features extracted either from a single data source (HSI) or from multiple sources that we call generically multiviews of the same scene. An important novelty of our approach is that it integrates local and nonlocal spatial information from each view in a unified framework. Our model learns a common intrinsic cluster structure from view-specific subspace representations by a new decomposition-based scheme. In addition, we develop innovative manifold-based spatial regularization as a hybrid hypergraph, which merges local and nonlocal spatial context and improves, thereby, the learning of view-specific structures. We develop an efficient algorithm to solve the resulting optimization problem. Extensive experiments on real data sets demonstrate the superior clustering performance over the state of the art.

Band selection, which removes irrelevant bands from hyperspectral images (HSIs) and keeps essential spectral information contained in a relatively few bands, allows huge savings in data storage, computation time, and imaging hardware. In this article, we propose a novel structural subspace clustering (STSC) method for hyperspectral band selection, which leverages the self-representation property of data and structural prior information to learn the cluster structure of bands. Particularly, we propose a general clustering model where the coarse coefficients matrix derived from a self-representation model is decomposed as a combination of a desirable coefficients matrix and a sparse matrix. This strategy adaptively adjusts the coarse coefficients matrix to learn the intrinsic data structure in low-dimensional subspaces. To guide this learning process, we introduce a structural regularization approach which makes use of the prior information about local and global properties of spectral bands. Moreover, we incorporate also prior knowledge about the dictionary, which demonstrates to yield a better clustering performance. We develop an adaptive method to estimate the number of selected bands by analyzing eigenvalue gaps of Laplacian matrix. To solve the resulting model, an efficient algorithm based on alternating direction method of multipliers (ADMMs) is developed. Extensive experiments on benchmark HSIs show that our method outperforms the state-of-the-art band selection methods.

Sparse subspace clustering (SSC) has emerged as an effective approach for the automatic analysis of hyperspectral images (HSI). Traditional SSC-based approaches employ the input HSI data as a dictionary of atoms, in terms of which all the data samples are linearly represented. This leads to highly redundant dictionaries of huge size, and the computational complexity of the resulting optimization problems becomes prohibitive for large-scale data. In this article, we propose a scalable subspace clustering method, which integrates the learning of a concise dictionary and robust subspace representation in a unified model. This reduces significantly the size of the involved optimization problems. We introduce a new adaptive spatial regularization for the representation coefficients, which incorporates spatial information of HSI and improves the robustness of the model to noise. We derive an effective solver based on alternating minimization and alternating direction method of multipliers (ADMMs) to solve the resulting optimization problem. Experimental results on four representative hyperspectral images show the effectiveness of the proposed method and excellent clustering performance relative to the state of the art.

Existing methods for tensor completion (TC) have limited ability for characterizing low-rank (LR) structures. To depict the complex hierarchical knowledge with implicit sparsity attributes hidden in a tensor, we propose a new multilayer sparsity-based tensor decomposition (MLSTD) for the low-rank tensor completion (LRTC). The method encodes the structured sparsity of a tensor by the multiple-layer representation. Specifically, we use the CANDECOMP/PARAFAC (CP) model to decompose a tensor into an ensemble of the sum of rank-1 tensors, and the number of rank-1 components is easily interpreted as the first-layer sparsity measure. Presumably, the factor matrices are smooth since local piecewise property exists in within-mode correlation. In subspace, the local smoothness can be regarded as the second-layer sparsity. To describe the refined structures of factor/subspace sparsity, we introduce a new sparsity insight of subspace smoothness: a self-adaptive low-rank matrix factorization (LRMF) scheme, called the third-layer sparsity. By the progressive description of the sparsity structure, we formulate an MLSTD model and embed it into the LRTC problem. Then, an effective alternating direction method of multipliers (ADMM) algorithm is designed for the MLSTD minimization problem. Various experiments in RGB images, hyperspectral images (HSIs), and videos substantiate that the proposed LRTC methods are superior to state-of-the-art methods.

Sparse subspace clustering (SSC) has achieved the state-of-the-art performance in clustering of hyperspectral images. However, the computational complexity of SSC-based methods is prohibitive for large-scale problems. We propose a large-scale SSC-based method, which processes efficiently large-scale HSIs without sacrificing the clustering accuracy. The proposed approach incorporates sketching of the self-representation dictionary reducing thereby largely the number of optimization variables. In addition, we employ a total variation (TV) regularization of the sparse matrix, resulting in a robust sparse representation. We derive a solver based on the alternating direction method of multipliers (ADMM) for the resulting optimization problem. Experimental results on real data show improvements over the traditional SSC-based methods in terms of accuracy and running time.

Sparse representation based methods have demonstrated their superior performance in target detection tasks compared to more traditional approaches such as matched subspace detectors and adaptive subspace detectors. However, the existing sparsity-based target detection methods were mostly formulated for and validated on a single imaging modality (sometimes with multiple spectral bands). In many application domains, including art investigation, multimodal data, acquired by different sensors are readily available, and yet, efficient processing techniques for such data are still scarce. In this paper, we propose a sparsity-based multimodal target detection method that processes jointly the information from multiple imaging modalities in a kernel feature space, and making use of the spatial context. We develop our target detector such to be robust to errors in labelled data, which is especially important in applications like digital painting analysis, where pixel-wise manual annotations are unreliable. We apply the proposed method to a challenging application of paint loss detection in master paintings and we demonstrate its effectiveness on a case study with multimodal acquisitions of the Ghent Altarpiece.

Sparse subspace clustering (SSC) has achieved the state-of-the-art performance in the clustering of hyperspectral images (HSIs). However, the high computational complexity and sensitivity to noise limit its clustering performance. In this paper, we propose a scalable SSC method for the large-scale HSIs, which significantly accelerates the clustering speed of SSC without sacrificing clustering accuracy. A small landmark dictionary is first generated by applying k-means to the original data, which results in the significant reduction of the number of optimization variables in terms of sparse matrix. In addition, we incorporate spatial reg-ularization based on total variation (TV) and improve this way strongly robustness to noise. A landmark-based spectral clustering method is applied to the obtained sparse matrix, which further improves the clustering speed. Experimental results on two real HSIs demonstrate the effectiveness of the proposed method and the superior performance compared to both traditional SSC-based methods and the related large-scale clustering methods.

Sparse subspace clustering (SSC) has been widely applied in remote sensing demonstrating excellent performance. Recent extensions incorporate spatial information, typically via smoothness-enforcing regularization. We propose an alternative approach: a joint sparsity SSC model, where pixels within a local region are enforced to select a common set of samples in the subspace-sparse representation. The corresponding optimization problem is solved by the alternating direction method of multipliers (ADMM). Experimental results on real data show a significant improvement over SSC and related state-of-the-art methods.

Sparsity-based classification methods have been widely used in hyperspectral image (HSI) classification. These methods typically assumed Gaussian noise, neglecting the fact that HSIs are often corrupted by different types of noise in practice. In this paper, we develop a robust super-pixel level joint sparse representation classification model (RSJSRC) to address the mixed noise problem in sparsity-based HSI classification. Our method takes into account both Gaussian and sparse noise. Experimental results on simulated and real data demonstrate the efficiency of the proposed method and clear benefits from the introduced mixed-noise model.

A method for stereo matching using topological information of stereo images is described. Two topics are discussed: topological information of images and their computation, and a new cost function of the multidimensional dynamic programming (DP) in the matching process. Both low-level features and the topological information of images are incorporated in the DP function. The results with real images show that the topological information as global restriction for matching improves the reliability of the matching processing.>