Small target detection in remote sensing images is a significiant reserach focus within the remote sensing domain. Recently, various YOLO algorithms have demonstrated remarkable achievements in the detection of small targets in remote sensing. However, YOLO-based detection algorithms still face challenges in this context, including limited feature expression capacity, difficulties in mitigating aliasing effects and inadequate adaptability to complex-shaped targets. To address these issues, this paper proposes a novel object detection network SOD-YOLOv8n. First, we propose a novel multi-path feature fusion module (MFFM), which enhances feature extraction through diverse dimensional feature processing strategies (global, local, channel, and spatial). It also fuses complementary information across channels via channel shuffling, thereby augmenting feature representation capabilities, and boosting the detection accuracy of small targets in remote sensing. Secondly, we design an anti-aliasing module (AAM) that employs wavelet pooling technology for frequency decomposition to mitigate the aliasing effect generated during model downsampling, thereby better retaining the key high-frequency information of small targets in remote sensing. Finally, we introduce the Shape-IoU loss function, which emphasizes the edge features of the target shape (such as contours, curvature, etc.) by calculating the similarity between the true target and the target shape in the predicted box, so as to better match targets with complex shapes.We conduct extensive experiments on the AI-TOD and USOD remote sensing small target datasets, and the results show that SOD-YOLOv8n outperforms several established state-of-the-art detection models.

In the field of salient object detection (SOD), optical remote sensing images (ORSI) differ significantly from natural sensing images (NSI). Existing research in ORSI-based SOD is constrained by the limitations of convolutional neural networks (CNNs) in feature extraction and by the underutilization of feature information in Transformer-based approaches. To address these challenges, this paper presents a Transformer-based Local-Global Information Perception Network (LGIPNet) for ORSI, which enhances encoder-generated features at multiple levels to highlight salient targets through three specialized feature enhancement modules. The Edge Adaptive Enhancement Module (EAEM) focuses on extracting local edge features to guide precise edge generation. The Multi-scale Grouped Weighted Attention Module (MGWAM) extracts local information from low-level features, scales features, and uses multi-scale channel and learnable weighted spatial attention to locate salient targets. For high-level features, the Dual-Domain Attention Module (DDAM) integrates a Channel Enhancement Attention Block (CEA) and an Adaptive Spatial Attention Block (ASA) to refine both local and global information. Specifically, the EAEM sharpens the edges of salient objects, thereby ensuring the precision of boundary detection. The MGWAM, on the other hand, enriches the feature representation across multiple scales, enhancing the network’s capability to encapsulate both fine-grained details and broader contextual information. The DDAM further strengthens the balance between local and global information, preserving feature integrity across levels. Finally, multi-scale features are cascaded to produce the final saliency map. This holistic strategy empowers LGIPNet to accurately identify and emphasize salient objects in ORSI. Experiments on three datasets demonstrate that LGIPNet outperforms existing state-of-the-art methods, establishing its effectiveness and robustness in ORSI-based SOD. The source code is available at https://github.com/sCauliflower/LGIPNet.git.

The detection of remote sensing (RS) objects plays a crucial role in various earth observation tasks. Current RS object detection methods tend to face great challenges due to large-scale variations of object sizes and limited representation of semantic features, which affect the final bounding box regression results in practice. To address the challenges, we propose a novel Multi-branch and Multi-level Feature Enhancement framework (M2FE-YOLO) to refine feature representation learning for improving the performance of object detection. The proposed M2FE-YOLO primarily comprises three components, i.e., Multi-Branch Feature-awareness based CSP (MBFA-CSP) module, Multi-Level Feature Fusion (MLFF) module, and Shape-IoU as a loss function. MBFA-CSP builds on a dual-scale feature-aware mechanism and a serial convolution pathway to dynamically adjust receptive fields, capturing critical contextual patterns in RS images. MLFF resolves the persistent semantic discrepancy between low-level texture features and high-level abstract representations, enabling precise localization of objects in cluttered RS landscapes. Shape-IoU instead of C-IoU incorporates a geometric compatibility factor (i.e., aspect ratio consistency), and is more crucial for bounding box regression of elongated or irregular RS objects. Compared with existing state-of-the-art methods, extensive experiments on RSOD, NWPU VHR-10, and DOTA datasets quantitatively and qualitatively demonstrate the superiority of the proposed M2FE-YOLO method, achieving up to 92.1%, 93.3%, and 72.2% mAP, respectively. Meanwhile, M2FE-YOLO-OBB achieves an excellent detection result of 73.9% mAP on DOTA dataset for oriented object detection task.

Vision Transformer (ViT), known for capturing non-local features, is an effective tool for hyperspectral image classification (HSIC). However, ViT's multi-head self-attention (MHSA) mechanism often struggles to balance local details and long-range relationships for complex high-dimensional data, leading to a loss in spectral-spatial information representation. To address this issue, we propose a deformable convolution-enhanced hierarchical Transformer with spectral-spatial cluster attention (SClusterFormer) for HSIC. The model incorporates a unique cluster attention mechanism that utilizes spectral angle similarity and Euclidean distance metrics to enhance the representation of fine-grained homogenous local details and improve discrimination of non-local structures in 3-D HSI and 2-D morphological data, respectively. Additionally, a dual-branch multiscale deformable convolution framework augmented with frequency-based spectral attention is designed to capture both the discrepancy patterns in high-frequency and overall trend of the spectral profile in low-frequency. Finally, we utilize a cross-feature pixel-level fusion module for collaborative cross-learning and fusion of the results from the dual-branch framework. Comprehensive experiments conducted on multiple HSIC datasets validate the superiority of our proposed SClusterFormer model, which outperforms existing methods. The source code of SClusterFormer is available at https://github.com/Fang666666/HSIC SClusterFormer.

The classification of land cover based on spectral-spatial joint features is a focus in the current domain of hyperspectral image (HSI) classification. However, existing methods might not fully leverage both spectral and spatial characteristics. Additionally, approaches that separately extract and later fuse spectral and spatial features often encounter challenges with suboptimal fusion outcomes. To address these issues, we propose a spectral-spatial dual-branch cross-enhanced transformer method. This approach first utilizes distinct shallow convolutional modules tailored to the attributes of spectral and spatial data for feature extraction. Subsequently, a cross-attention strategy is designed to better align spectral and spatial features and smoothly merge these features through attention operations. Furthermore, by introducing the fused spectral-spatial features, we extend the traditional multi-head self-attention (MHSA) mechanism. This richness in feature sequences aids the model in better capturing long-range dependencies between features. The experimental results on two benchmark datasets validate the superiority of the proposed method.

The effective combination of hyperspectral image (HSI) and light detection and ranging (LiDAR) data can be utilized for land cover classification. Recently, deep learning-based classification methods, especially those utilizing Transformer networks, have achieved remarkable success. However, deep learning classification methods for multi-source data still encounter various technical challenges, such as the comprehensive utilization of multi-scale information, the lightweight network design, and the efficient fusion strategies for heterogeneous data. To address these challenges, we propose a novel and efficient deep neural network, namely multi-scale 3D-2D mixed CNN feature extraction and multi-source data lightweight attention-free fusion network (M2FNet) based on CNN and Transformer. Through end-to-end training, this network effectively combines heterogeneous information from multiple sources, leading to improved performance in joint classification. Specifically, M2FNet employs a multi-scale 3D-2D mixed CNN design to extract both the spatial-spectral features of HSI and the depth-based elevation features of LiDAR data. Subsequently, the extracted features are fed into a novel encoder comprising a feature enhancement module, designed with mathematical morphology and a dilated convolutional module derived from the self-attention of the conventional Transformer encoder (DConvformer), which plays a crucial role in integrating multi-source information within the network. The well-designed architecture enables the network to acquire multi-scale depth and high-order features, significantly reducing the number of training parameters. Comparative experimental results and ablation studies demonstrate that M2FNet outperforms other advanced methods. The source code is publicly available at https://github.com/cupid6868/M2FNet.git.

The spatial resolution of hyperspectral images (HSIs) is usually limited due to internal imaging mechanisms. To obtain imagery with high spectral and high spatial resolutions, which is essential for subsequent HSI processing tasks, a cost-effective approach is to fuse HSI with multispectral images (MSIs). One highly effective fusion method is the convolutional neural network (CNN). However, CNNs have limitations in capturing global information and complex features. Recently, visual transformers (ViTs) have garnered interest for their ability to process non-local information. Despite this, existing HSI-MSI fusion methods suffer from insufficient spatial-spectral feature interaction, resulting in suboptimal fusion quality. To address these challenges, we propose a multiscale deep cross-fusion transformer (MDC-FusFormer) network for HSI and MSI fusion. This network effectively performs the interactive fusion of spatial-spectral features, thereby enhancing the quality of the fused images. MDC-FusFormer employs a three-branch network architecture consisting of two independent progressive feature mining modules (PFMMs), a multiscale deep cross-fusion attention module, and a spatial-spectral feature fusion module. Initially, shallow features at different scales of MSI and HSI are recursively extracted through successive up- and down-sampling using CNNs. These features then interact with the deep cross-modal information at corresponding scales through the attention block. Finally, a multidimensional refinement convolution block (MRCB) is applied to refine the feature information, which is then combined with cascaded up-sampling to reconstruct the high-resolution fused image step by step. Experimental results on five datasets indicate that, compared to nine other methods, MDC-FusFormer delivers superior performance.

In recent years, convolutional neural networks (CNNs) have achieved remarkable success in hyperspectral image (HSI) classification tasks, primarily due to their outstanding spatial feature extraction capabilities. However, CNNs struggle to capture the diagnostic spectral information inherent in HSI. In contrast, vision transformers exhibit formidable prowess in handling spectral sequence information and excelling at capturing long-range correlations between pixels and bands. Nevertheless, due to the information loss during propagation, some existing transformer-based classification methods struggle to form sufficient spectral-spatial information mixing. To mitigate these limitations, we propose a memory-augmented spectral-spatial transformer (MASSFormer) for HSI classification. Specifically, MASSFormer incorporates two efficacious modules, the memory tokenizer (MT) and the memory-augmented transformer encoder (MATE). The former serves to transform spectral-spatial features into memory tokens for storing prior knowledge. The latter aims to extend traditional multi-head self-attention (MHSA) operations by incorporating these memory tokens, enabling ample information blending while alleviating the potential depth decay in the model, and consequently improving the model’s classification performance. Extensive experiments conducted on four benchmark datasets demonstrate that the proposed method outperforms state-of-the-art methods. The source code is available at https://github.com/hz63/MASSFormer for the sake of reproducibility.

Recently, tensor singular value decomposition (t-SVD) has demonstrated excellent performance in various high-dimensional information processing applications. However, in adapting t-SVD to handle the typical tensor data restoration tasks, such as hyperspectral image (HSI) denoising, the following questions remain inadequately addressed: 1) The existing tensor nuclear norm minimization (TNN) regime treats all tensor singular values alike; thus, it lacks flexibility and dominance in dealing with the sophisticated HSI tensor. 2) The existing t-SVD-based denoising methods can not directly process order-p(p> 3) tensors; thus, they fail to comprehensively exploit the high-dimensional structural correlation of the HSI tensor along different modes. To address the above challenges, in this study, we first generalize a novel weighted order-pTNN minimization regime, which integrates the adaptively reweighting strategy for matrix, third-order, and order-ptensors in a unified architecture. Subsequently, an efficient subspace low-rank learning model is established, using HSI denoising tasks as an application example to corroborate the superiority of the proposed regime in approximating the high-dimensional low-rank structure of natural tensor data. Extensive experimental results substantiate that our effort surpasses existing state-of-the-art low-rank tensor recovery methods in both restoration accuracy and efficiency. The source code is available at https://github.com/CX-He/WTNN.git.

In hyperspectral images (HSIs), mixed noise (e.g., Gaussian noise, impulse noise, stripe noise, and deadlines) contamination is a common phenomenon that greatly reduces the visual quality of the image. In recent years, methods combining global and non-local low-rankness have been widely used in the field of HSI denoising. However, most methods apply original space-based denoising strategies (low-rank tensor decomposition, total variation, and tensor sparse representation, etc.) directly to the modeling of non-local low-rank tensors in subspace, without fully exploiting the intrinsic and latent properties of the non-local similar tensors. In this paper, we propose a hybrid prior denoising method based on global tensor low-rankness and non-local SVD-aided group sparsity (GTL_NSGS). This method introduces a novel plug-and-play NSGS denoiser that uses singular value decomposition as assistance to successively explore self-similarity of spatial dimension, low-rankness of spectral dimension, and group sparsity of difference domain in subspace non-local similar tensors. Globally, we utilize the existing three-way log-based tensor nuclear norm (3DLogTNN) to approximate the HSI tensor fibered rank and introduce a difference continuity regularization to obtain a continuous smooth spectral basis. Finally, we combine the Alternating Direction Method of Multipliers (ADMM) with the Augmented Lagrangian Multiplication (ALM) algorithm to solve the proposed model effectively. Extensive experiments on simulated and real data sets demonstrate that the proposed method has superior performance in removing mixed noise compared to state-of-the-art denoisers.

Hyperspectral image (HSI) classification is currently a hot topic in the field of remote sensing. The goal is to utilize the spectral and spatial information from HSI to accurately identify land covers. Convolution neural network (CNN) is a powerful approach for HSI classification. However, CNN has limited ability to capture non-local information to represent complex features. Recently, vision transformers (ViTs) have gained attention due to their ability to process non-local information. Yet, under the HSI classification scenario with ultra-small sample rates, the spectral-spatial information given to ViTs for global modeling is insufficient, resulting in limited classification capability. Therefore, in this article, Multi-Attention Joint Convolution Feature Representation with Lightweight Transformer (MAR-LWFormer) is proposed, which effectively combines the spectral and spatial features of HSI to achieve efficient classification performance at ultra-small sample rates. Specifically, we use a three-branch network architecture to extract multi-scale convolved 3D-CNN, EMAP, and LBP features of HSI, respectively, by taking full exploitation of ultra-small training samples. Second, we design a series of multi-attention modules to enhance spectral-spatial representation for the three types of features and to improve the coupling and fusion of multiple features. Third, we propose an explicit feature attention tokenizer to transform the feature information, which maximizes the effective spectral-spatial information retained in the flat tokens. Finally, the generated tokens are input to the designed lightweight transformer for encoding and classification. Experimental results on three datasets validate that MAR-LWFormer has an excellent performance in HSI classification at ultra-small sample rates when compared to several state-of-the-art classifiers.

Despite the remarkable progress made by the salient object detection of natural sensing images (NSI-SOD), the complex background and scale diversity issues of remote sensing images (RSIs) still pose a substantial obstacle. In this study, we build an end-to-end channel-enhanced remodeling-based network (CRNet) for optical RSIs (ORSIs) to highlight salient objects through feature augmentation. First, the backbone convolutional block is used to suggest the fundamental characteristics. Then, we use the channel enhance module (CEM) to enhance the shallow features. CEM primarily relies on the channel attention mechanism and employs a no-downscaling strategy to produce local cross-channel interaction, which lowers model complexity while enhancing extraction performance. Meanwhile, we use the redefined feature module (RFM) to reconstruct the deep features and generate global attention features by dimensional transformation and feature relationship aggregation to achieve the role of locating salient targets. Finally, the cascade combines the multi-scale features to provide the final saliency map. To further enhance the representational power of the network, we use a hybrid loss function to improve performance. The proposed approach outperforms current state-of-the-art methods, as shown by several experiments on three available datasets. The source code of the proposed CRNet is available publicly at https://github.com/hilitteq/CRNet.git.

The joint use of multisource remote-sensing (RS) data for Earth observation missions has drawn much attention. Although the fusion of several data sources can improve the accuracy of land-cover identification, many technical obstacles, such as disparate data structures, irrelevant physical characteristics, and a lack of training data, exist. In this article, a novel dual-branch method, consisting of a hierarchical convolutional neural network (CNN) and a transformer network, is proposed for fusing multisource heterogeneous information and improving joint classification performance. First, by combining the CNN with a transformer, the proposed dual-branch network can significantly capture and learn spectral–spatial features from hyperspectral image (HSI) data and elevation features from light detection and ranging (LiDAR) data. Then, to fuse these two sets of data features, a cross-token attention (CTA) fusion encoder is designed in a specialty. The well-designed deep hierarchical architecture takes full advantage of the powerful spatial context information extraction ability of the CNN and the strong long-range dependency modeling ability of the transformer network based on the self-attention (SA) mechanism. Four standard datasets are used in experiments to verify the effectiveness of the approach. The experimental results reveal that the proposed framework can perform noticeably better than state-of-the-art methods. The source code of the proposed method will be available publicly athttps://github.com/zgr6010/Fusion_HCT.git.

Low-rank tensor representation philosophy has enjoyed a reputation in many hyperspectral image (HSI) low-level vision applications, but previous studies often failed to comprehensively exploit the low-rank nature of HSI along different modes in low-dimensional subspace, and unsurprisingly handled only one specific task. To address these challenges, in this paper, we figured out that in addition to the spatial correlation, the spectral dependency of HSI also implicitly exists in the coefficient tensor of its subspace, this crucial dependency that was not fully utilized by previous studies yet can be effectively exploited in a cascaded manner. This led us to propose a unified subspace low-rank learning regime with a new tensor cascaded rank minimization, named STCR, to fully couple the low-rankness of HSI in different domains for various low-level vision tasks. Technically, the high-dimensional HSI was first projected into a low-dimensional tensor subspace, then a novel tensor low-cascaded-rank decomposition was designed to collapse the constructed tensor into three core tensors in succession to more thoroughly exploit the correlations in spatial, nonlocal, and spectral modes of the coefficient tensor. Next, difference continuity-regularization was introduced to learn a basis that more closely approximates the HSI's endmembers. The proposed regime realizes a comprehensive delineation of the self-portrait of HSI tensor. Extensive evaluations conducted with dozens of state-of-the-art (SOTA) baselines on eight datasets verified that the proposed regime is highly effective and robust to typical HSI low-level vision tasks, including denoising, compressive sensing reconstruction, inpainting, and destriping. The source code of our method is released at https://github.com/CX-He/STCR.git.

Video stabilization is the process of improving the video quality by removing annoying fluctuant motion caused by camera jittering. A key issue of a successful solution is the temporal adaptability to motion and the overall robustness with respect to different motion types. However, most previous methods usually produce non-motion adaptive stabilized videos. In other words, under-smoothing in slow motion segments and over-smoothing in rapid motion segments will be produced for complex shaky videos. To overcome these drawbacks, we propose a novel video stabilization approach using a motion morphological component (MMC) decomposition. Specifically, the observed motion is decomposed into three MMCs: low-frequency smoothed (LFS) motion, high-frequency compensatory (HFC) motion, and shaky motion. LFS motion helps to largely stabilize videos, and HFC motion helps to recover missing motion to deal with over-smoothing. Subsequently, we present an MMC-based model to retrieve the desired smoothed motion, in which weighted nuclear norm and autoregression priors are used for LFS motion, while a sparsity prior is adopted for HFC motion. In addition, we design an adaptive weight setting scheme to detect rapid motions and to calculate the optimal weights. Finally, we develop a stabilization algorithm under the Alternating Direction Method of Multipliers (ADMM) framework. Experimental results demonstrate that our method can achieve high-quality results compared with that of other state-of-the-art stabilization methods in terms of robustness and efficiency, both quantitatively and qualitatively.

Most of the traditional hyperspectral unmixing methods are based on the matrix and often ignore the spatial information of hyperspectral images (HSIs). In recent years, tensor-based methods have been gradually used in hyperspectral unmixing, owing to their ability to completely preserve the real spatial structure of HSIs. A blind unmixing method for HSIs based on an$L_{1}$regular term and tucker tensor decomposition (BUTTDL1) is proposed, which describes the low rank of abundance by tucker tensor decomposition in the form of a third-order tensor and increases the sparse characterization of abundance by the$L_{1}$norm. A comparative experiment is performed on two simulation datasets. Compared with the latest method unmixing with low-rank tensor regularization algorithm accounting for endmembers variability (ULTRA-V), in the simulation dataset Data Cube 1 (DC1), the endmember mean square error (MSE) of BUTTDL1 is decreased by 1.1, and the abundance MSE is decreased by 8.6. In the simulation dataset DC2, the endmember MSE is decreased by 2.4, and the abundance MSE is decreased by 6.63.

With the rapid advancement of spectrometers, the imaging range of the electromagnetic spectrum starts growing narrower. The reduction of electromagnetic wave energy received in a single wavelength range leads more complex noise into the generated hyperspectral image (HSI), thus causing a severe cripple in the accuracy of subsequent applications. The requirement for the HSI mixed denoising algorithm’s accuracy is further lifted. To address this challenge, in this letter, we propose a novel difference continuity-regularized nonlocal tensor subspace low-rank learning (named DNTSLR) method for HSI mixed denoising. Technically, the original high-dimensional HSI data was first projected into a low-dimensional subspace spanned by a spectral difference continuous basis instead of an orthogonal basis, so the data continuity of the restored HSI spectrum and tensor low-rankness was guaranteed. Then, a cube matching strategy was employed to stack the nonlocal tensor patches from the projected coefficient tensor, and a shrinkage algorithm was used to approximate the low-rank coefficient tensor. Eventually, the subspace low-rank learning algorithm was designed to alternately separate the noise tensor and restore the latent clean low-rank HSI tensor. Extensive experiments on multiple open datasets validate that the proposed method realizes the state-of-the-art denoising accuracy for HSI.

In this letter, using the sparse unmixing framework, a weighted collaborative sparse and$L_{1/2}$low-rank regularization with superpixel segmentation method is proposed for hyperspectral unmixing. The method outlined here first uses superpixel segmentation to obtain local homogeneous regions. The reason for this approach is that the shape and size of superpixels are adaptive, which are better for obtaining homogeneous regions than square patches. Next, the weighted collaborative sparse term and$L_{1/2}$low-rank regularization were utilized to exploit the spatial and spectral correlation of each superpixel. In addition, the smoothness between adjacent pixels is enforced by total variation regularization. Finally, the proposed method and several state-of-the-art methods were tested on two simulated data sets and two real data sets. The results demonstrate the superiority of the method proposed here.

Hyperspectral image (HSI) classification refers to accurately corresponding each pixel in an HSI to a land-cover label. Recently, the successful application of multiscale and multifeature methods has greatly improved the performance of HSI classification due to their enhanced utilization of the available spectral–spatial information. However, as the number of scales and the number of features increases, it becomes more difficult to achieve an optimal degree of fusion for multiple classifiers [e.g., kernel extreme learning machine (KELM)]. On the other hand, a limited sample size of the HSI may cause overfitting problems, which seriously affects the classification accuracy. Therefore, in this article, a novel multi-structure KELM with attention fusion strategy (MSAF-KELM) is proposed to achieve accurate fusion of multiple classifiers for effective HSI classification with ultrasmall sample rates. First, a multi-structure network is built, which combines multiple scales and multiple features to extract abundant spectral–spatial information. Second, a fast and efficient KELM is employed to enable rapid classification. Finally, a weighted self-attention fusion strategy (WSAFS) is introduced, which combines the output weights of each KELM subbranch and the self-attention mechanism to achieve an efficient fusion result on multi-structure networks. We conducted experiments on four types of HSI datasets with different evaluation methods and compared them with several classical and state-of-the-art methods, which demonstrate the excellent performance of our method on ultrasmall sample rates. The code is available athttps://github.com/Fang666666/MSAF-KELMfor reproducibility.

In hyperspectral image (HSI) classification, each pixel sample is assigned to a land-cover category. In the recent past, convolutional neural network (CNN)-based HSI classification methods have greatly improved performance due to their superior ability to represent features. However, these methods have limited ability to obtain deep semantic features, and as the layer’s number increases, computational costs rise significantly. The transformer framework can represent high-level semantic features well. In this article, a spectral–spatial feature tokenization transformer (SSFTT) method is proposed to capture spectral–spatial features and high-level semantic features. First, a spectral–spatial feature extraction module is built to extract low-level features. This module is composed of a 3-D convolution layer and a 2-D convolution layer, which are used to extract the shallow spectral and spatial features. Second, a Gaussian weighted feature tokenizer is introduced for features transformation. Third, the transformed features are input into the transformer encoder module for feature representation and learning. Finally, a linear layer is used to identify the first learnable token to obtain the sample label. Using three standard datasets, experimental analysis confirms that the computation time is less than other deep learning methods and the performance of the classification outperforms several current state-of-the-art methods. The code of this work is available athttps://github.com/zgr6010/HSI_SSFTTfor the sake of reproducibility.

Kernel methods, e.g., composite kernels (CKs) and spatial-spectral kernels (SSKs), have been demonstrated to be an effective way to exploit the spatial-spectral information nonlinearly for improving the classification performance of hyperspectral image (HSI). However, these methods are always conducted with square-shaped window or superpixel techniques. Both techniques are likely to misclassify the pixels that lie at the boundaries of class, and thus a small target is always smoothed away. To alleviate these problems, in this paper, we propose a novel patch-based low rank component induced spatial-spectral kernel method, termed LRCISSK, for HSI classification. First, the latent low-rank features of spectra in each cubic patch of HSI are reconstructed by a low rank matrix recovery (LRMR) technique, and then, to further explore more accurate spatial information, they are used to identify a homogeneous neighborhood for the target pixel (i.e., the centroid pixel) adaptively. Finally, the adaptively identified homogenous neighborhood which consists of the latent low-rank spectra is embedded into the spatial-spectral kernel framework. It can easily map the spectra into the nonlinearly complex manifolds and enable a classifier (e.g., support vector machine, SVM) to distinguish them effectively. Experimental results on three real HSI datasets validate that the proposed LRCISSK method can effectively explore the spatial-spectral information and deliver superior performance with at least 1.30% higher OA and 1.03% higher AA on average when compared to other state-of-the-art classifiers.

Structural support vector machine (SSVM) is popular in the visual tracking field as it provides a consistent target representation for both learning and detection. However, the spatial distribution of feature is not considered in standard SSVM-based trackers, therefore leading to limited performance. To obtain a robust discriminative classifier, this paper proposes a novel tracking framework that spatially regularizes SSVM, which yields a new spatially regularized SSVM (SRSSVM). We utilize the spatial regularization prior to penalize the learning classifier with the same size as the target region. The location of classifier spatially located far from the center of region is assigned large weight and vice versa. Then, it is introduced into the SSVM model as a regularization factor to learn the robust discriminative model. Furthermore, an optimizing algorithm with dual coordination descent is presented to efficiently solve the SRSSVM tracking model. Our proposed SRSSVM tracking method has low computational cost like the traditional linear SSVM tracker while can significantly improve the robustness of the discriminative classifier. The experimental results on three popular tracking benchmark data sets show that the proposed SRSSVM tracking method performs favorably against the state-of-the-art trackers.

In this paper, we propose a novel cross total variation regularized unidirectional nonlocal low rank tensor approximation method for hyperspectral image denoising. It fully explores the spectral-spatial correlation and non-local self-similarity simultaneously in tensor case and points out that the nonlocal self-similarity is the most important for precisely restoring the HSI. Following the research line in [1], we propose to embed the cross total variation (CrTV) regularization into the unidirectional low rank tensor framework to alleviate the common consistency issue of pixels in overlapped regions. CrTV shows great power to explore the spatial-spectral correlation and has great ability to keep the fine spatial details and preserve the spectra in the course of HSI denoising. The final model can be effectively solved by the alternating direction methods of multipliers (ADMM). Experimental results on HSI data sets validate that the complementary priors (i.e., spatial-spectral correlation and non local self-similarity) really contribute to the performance and also illustrate the superiority of the proposed method when compared with other state-of-the-art denoising methods.

Generally, the traditional semi-supervised extreme learning machine (S2-ELM) method cannot fully exploit the limited label information in hyperspectral image (HSI) classification. In this paper, we propose a discriminative S2-ELM method, called pixel-pairwise constrained S2-ELM (P2S2- ELM) method. Both the manifold regularization to leverage unlabeled data and the pixel-pairwise constraint between the labeled pixels are incorporated into a unified minimizing framework, thus the proposed P2S2-ELM method is able to learn a more effective and discriminative projection. Experimental results on several real hyperspectral data sets exhibit its efficiency and superiority to the counterparts, when only a small number of labeled samples are available.

This paper proposes a novel mixed noise removal method via subspace low rank representation and BM4D filtering for hyperspectral imagery (HSI). The proposed method is based on the following two facts. The first one is that the spectra in each class of HSI lie in different low-rank subspace, that is, the HSI data could be decomposed into two sub-matrices with lower ranks in the framework of subspace low rank representation. The second one is that the spatial structures of HSI have the property of non-local self-similarity (NSS), and the NSS could be effectively exploited by BM4D filter with no additional parameters. The proposed model can be easily and effectively solved by splitting it into several sub-problems via the alternating direction method of multipliers (ADMM). Experimental results validate that the proposed method outperforms other state-of-the-art denoising methods for HSI.

Recently, the hidden Markov model (HMM) with student’s t-mixture model (SMM), called student’s t-HMM (SHMM) for short, has received much attention in unsupervised learning of sequential data. However, the current existing SHMMs fail to take into consideration of the relevant features embedded in local subspaces, thus influencing their performances in clustering. To address the problem, a novel SHMM is proposed by combining the measure of localized feature saliency (LFS) with SMM and utilizing two student’s t-distributions as subcomponents to respectively describe the distributions of useful features and non-salient “features,” with the purpose of accurately modeling the hidden state observation emission distributions of SHMM. Moreover, we exploit the variational Bayesian learning technique to simultaneously estimate the LFS, the number of components and other parameters of the herein proposed SHMM. Experimental results on both synthetic and real data sets demonstrate the improved robustness, effectiveness, and accuracy of our model.

In this paper, a new classifier under Bayesian framework is proposed to explore homogeneous region based low rank representation in hidden field for classification of hyperspectral imagery (HSI). This classifier integrates low rank representation and superpixel segmentation simultaneously, in which the HSI data is assumed to be lying in a low rank subspace within each homogeneous region of an estimated hidden field. First, the HSI data is projected into the Principal Component space, then the first principal component image is segmented into hundreds of homogeneous regions. Following, the spectral-only supervised Bayesian classifier, i.e., Sparse Multinomial Logistic Regression (SMLR), is utilized for estimating the likelihood probabilities of testing samples, then spatial information is exploited by low rank representation within each superpixel in a hidden field which is approximated to the pre-estimated likelihood probabilities. The proposed model can be easily solved by alternating direction method of multipliers (ADMM). Experimental results on real hyperspectral data, i.e., AVIRIS Indian Pines and ROSIS University of Pavia, show that the proposed classifier outperforms other state-of-the-art classifiers in terms of quantitative assessment and visual effect.

Due to the limitation of sensors and atmospheric conditions, hyperspectral images (HSI) are always contaminated by heavy noises, which significantly limits the subsequent applications. To mitigate the problem, this paper proposes a novel subspace spatial-spectral low rank learning method for hyper-spectral denoising. It is based on the assumption that spectra in HSI lie in a low-rank subspace and nonlocal spatial patches are self-similar. The spectral low-rank property is explored by decomposing the clean HSI into two sub-matrices of low rank and the spatial self similarity is exploited by weighed nuclear norm minimization in a nonlocal sense. The proposed restoration model is formulated into an iterative optimization model which can be effectively solved by a cyclic descent algorithm. Experimental results on both simulated and real HSI datasets show that the proposed method can significantly outperform the state-of-the-art methods in terms of quantitative assessment and visual quality.

This letter presents a novel mixed noise (i.e., Gaussian, impulse, stripe noises, or dead lines) reduction method for hyperspectral image (HSI) by utilizing low-rank representation (LRR) on spectral difference image. The proposed method is based on the assumption that all spectra in the spectral difference space of HSI lie in the same low-rank subspace. The LRR on the spectral difference space was exploited by nuclear norm of difference image along the spectral dimension. It showed great potential in removing structured sparse noise (e.g., stripes or dead lines located at the same place of each band) and heavy Gaussian noise. To simultaneously solve the proposed model and reduce computational load, alternating direction method of multipliers was utilized to achieve robust reconstruction. The experimental results on both simulated and real HSI data sets validated that the proposed method outperformed many state-of-the-art methods in terms of quantitative assessment and visual quality.

This paper proposes a novel linear hyperspectral unmixing method based on l1-l2sparsity and total variation (TV) regularization. First, the enhanced sparsity based on l1-l2norm is explored to depict the intrinsic sparse characteristic of the fractional abundances in sparse regression unmixing model. By taking the correlation between hyperspectral pixels into account, total variation is minimized to enforce the spatial smoothness. Finally, the proposed model is solved by the extended alternating direction method of multipliers (ADMM). Experimental results on simulated and real hyperspectral datasets validate the excellent performances of the proposed method.

In this paper, we propose a new fractional order variational method for pan-sharpening, which aims to obtain a high resolution multi-spectral (MS) image from a low resolution MS image and a high resolution panchromatic (PAN) image. On one hand, we use the data generative constraint for preserving the spectral information. More specifically, on the other hand, we exploit the fractional order gradient feature consistence between the high resolution MS image and PAN image for preserving the spatial information. Based on these assumptions, a new fractional order variational model is proposed and an efficient algorithm is designed to solve the proposed model. Experimental results show that the proposed method outperforms various well-known pan-sharpening methods in terms of higher spatial and spectral qualities.

In this paper, we propose a new method for hyperspectral image (HSI) classification using multi-layer superpixel graph and loopy belief propagation. A merging algorithm using graph based representation of image is applied to generate multi-scale superpixels in hyperspectral image at first. Then, we build a multi-layer superpixel graph and use loopy belief propagation to transmit messages between the superpixels and compute beliefs at each superpixel in our multi-layer graph for HSI classification. Experimental results with real hyperspectral data set demonstrate that our proposed method provides good performance and is competitive with some of the best available spectral-spatial methods for hyperspectral image classification.

In this letter, a real-time implementation of the logistic regression via variable splitting and augmented Lagrangian (LORSAL) algorithm for sparse multinomial logistic regression is presented on commodity graphics processing units (GPUs) using Nvidia's compute unified device architecture. The proposed parallel method properly exploits the GPU architecture at the low level, including its shared memory, and takes full advantage of the computational power of GPUs to achieve real-time classification performance of hyperspectral images for the first time in the hyperspectral imaging literature. Our experimental results reveal remarkable acceleration factors and real-time performance, while retaining exactly the same classification accuracy with regard to the serial and multicore versions of the classifier.

This paper presents a new approach for hyperspectral image classification exploiting spectral-spatial information. Under the maximum a posteriori framework, we propose a supervised classification model which includes a spectral data fidelity term and a spatially adaptive Markov random field (MRF) prior in the hidden field. The data fidelity term adopted in this paper is learned from the sparse multinomial logistic regression (SMLR) classifier, while the spatially adaptive MRF prior is modeled by a spatially adaptive total variation (SpATV) regularization to enforce a spatially smooth classifier. To further improve the classification accuracy, the true labels of training samples are fixed as an additional constraint in the proposed model. Thus, our model takes full advantage of exploiting the spatial and contextual information present in the hyperspectral image. An efficient hyperspectral image classification algorithm, named SMLR-SpATV, is then developed to solve the final proposed model using the alternating direction method of multipliers. Experimental results on real hyperspectral data sets demonstrate that the proposed approach outperforms many state-of-the-art methods in terms of the overall accuracy, average accuracy, and kappa (k) statistic.

This letter presents a postprocessing algorithm for a kernel sparse representation (KSR)-based hyperspectral image classifier, which is based on the integration of spatial and spectral information. A pixelwise KSR is first used to find the sparse coefficient vectors of the hyperspectral image. Then, a sparsity concentration index (SCI) rule-guided semilocal spatial graph regularization (SSG), called SSG+SCI, is proposed to determine refined sparse coefficient vectors that promote spatial continuity within each class. Finally, these refined coefficient vectors are used to obtain the final classification map. Compared with previous approaches based on similar spatial-spectral postprocessing strategies, SSG+SCI clearly outperforms their results in terms of accuracy and the number of training samples, as it is demonstrated with two real hyperspectral images.

In this paper, we propose a new model for hyperspectral image classification using spectral-spatial information. The main contributions of our paper are that we exploit the posterior distribution from both spectral and spatial information in the original hyperspectral data. The association potential in our model is a sparse multinomial logistic regression (SMLR) classifier and the interaction potential is a spatial-relevant total variation (TV) constraint upon the posterior distribution itself which encourages neighboring pixels to belong to the same class. The proposed model is solved by the alternating direction method of multipliers (ADMM); we enhance the spatial smoothness by expanding the spatial information from the fixed labeled samples to the whole data to further improve the classification accuracy. Experimental results with real hyperspectral data set validate that our proposed approach provides good performance when compared with other state-of-the-art methods.

Hyperspectral unmixing is a typical problem of blind source separation, which can be solved by nonnegative matrix factorization (NMF). Sparsity based NMF will increase the efficiency of unmixing, but its computational complexity limits the possibility of utilizing it in time-critical applications. In this paper, method of parallel hyperspectral unmixing based on sparsity constrained nonnegative matrix factorization on Graphics Processing Units (CSNMF-GPU) is investigated and compared in terms of both accuracy and speed. The realization of the proposed method using Compute Unified Device Architecture (CUDA) on GPU are described and evaluated. The experimental results comparing with the serial implementations based on both simulated and real hyperspectral data demonstrate the effectiveness of the proposed parallel optimization approach.

Sparsity is an intrinsic property of hyperspectral images, which means that the collected pixels can be represented by a part of materials. In this paper, a new sparsity based method for hyperspectral unmixing is proposed, referred to as the constrained sparse nonnegative matrix factorization (CSNMF). First, a novel sparse term which is explored to measure the sparsity of hyperspectral images is introduced to restrict the abundances. Second, minimum distance constraint which is convex is applied to restrict the endmembers. Then the alternating direction method of multipliers (ADMM) is used to solve the proposed CSNMF. The experimental results based on both synthetic mixtures and a real image scene demonstrate the effectiveness of the proposed approach.