Abstract Does a query image with much higher resolution than that of the gallery image also affect the pedestrian re‐identification performance? If so, and how does it affect performance? The study proposes a novel framework for performing high‐resolution image reconstruction and pedestrian re‐identification tasks, independent of the query image resolution. More precisely, an end‐to‐end trainable Resolution Independent person Re‐identification network is proposed. It is composed of our designed Cross‐Resolution GAN and Embedding Batch Normalisation layers. The model is then compared with the traditional low‐resolution pedestrian recognition algorithm and the hybrid method of high‐resolution reconstruction and pedestrian re‐identification. The results demonstrate that the proposed method outperforms the state‐of‐the‐art methods in the pedestrian re‐identification task based on our expanded benchmark dataset. It also reaches an equivalent performance to the existing methods in the high‐resolution image reconstruction task.

Models based on low-rank and sparse decomposition (LRaSD) have been rapidly developed in the hyperspectral anomaly detection (HAD) task. However, traditional LRaSD models usually impose multiple complex regularizers on the background components, which inevitably increases the computational cost and fails to maximize the effectiveness between them. In addition, regularizers for abnormal components mostly penalize each pixel with the same intensity. To tackle these challenges, we propose a model based on enhanced 3-D total variation (TV) and sparse reweighted regularization, referred to as E-3DTVSR. Specifically, an enhanced 3-D TV (E-3DTV) regularization is adopted to simultaneously characterize the low-rank and piecewise smoothness of the background. Since E-3DTV applies sparse regularization to subspace base maps of gradient maps along all bands of a hyperspectral image (HSI), rather than the gradient maps themselves, this effectively removes noise and improves the detection efficiency of the model. Meanwhile, in order to enhance the sparsity of abnormal targets and distinguish sparse nonabnormal pixels, combined with the log-sum function and the reweighted$\ell _{1}$minimization strategy, a sparse reweighted regularization is designed to adaptively assign weight to each target. Experiments demonstrate that E-3DTVSR reaches the area under the ROC curve (AUC) scores of 99.86%, 99.59%, and 99.72% on three public HSI datasets, respectively, outperforming the current advanced approaches.

In recent years, tensor representation (TR) based hyperspectral anomaly detection approaches have attracted more and more attention. However, two urgent issues still need to be addressed: 1) existing tensor decomposition approaches for hyperspectral anomaly detection (HAD) cannot make full use of the spectral-spatial correlation of background components in hyperspectral images (HSIs); 2) most approaches based on TR overlook the piecewise-smooth of background components that exist simultaneously in the spectral and spatial domains. To this end, with the aid of an advanced multi-scale entanglement renormalization ansatz (MERA) tensor network, this paper proposes an algorithm based on MERA decomposition and enhanced total variation regularization (MERAETV) for HAD. Specifically, MERA decomposes the background tensor by contracting a top-level factor with the remaining semi-orthogonal and orthogonal factors. Due to the intricate interplay between semi-orthogonal (low-rank) and orthogonal factors, low-rank MERA approximation exhibits a robust representational capacity that effectively captures the spectral-spatial correlation of the background component. Meanwhile, an enhanced total variation (ETV) regularization is devised to capture the inherent piecewise-smooth of the background component in both spectral and spatial domains. Furthermore, our algorithm incorporates group sparsity constraint and Gaussian noise term to enhance the discrimination between anomalies and background. Finally, a highly efficient update scheme based on the alternating direction method of multipliers (ADMM) is designed. A large number of experiments on one synthetic and seven real HSIs demonstrate the superiority of our proposed approach.

Due to the poor quality image with low signal-to-clutter ratio (SCR), infrared (IR) small target detection is faced with great challenges in the remote sensing field. Despite the fact that the local contrast measure (LCM) has been widely applied for IR target detection, the existing LCM-based methods suffer from weak target detectability (TD) or background suppressibility (BS) in complicate background. In this paper, we propose a novel IR small target detection method based on improved tri-layer window local contrast measure (TrLCM). With an additional isolation circle in TrLCM, the influence of background on target detection is reduced to a certain extent. Besides, background suppressibility is also promoted through a designed adaptive adjustment coefficient. Comprehensive experiments and analysis on three datasets verify that the proposed TrLCM achieves advanced TD, BS and overall performance.

Some existing anomaly detection methods convert a 3-D hyperspectral data cube into a 2-D matrix, which inevitably destroys the spatial-spectral structure information of the hyperspectral data, resulting in the degradation of detection performance. In this paper, we propose a tensor low-rank sparse representation learning (TLRAD) method for hyperspectral anomaly detection (HAD), which can effectively maintain the spatial-spectral structure of raw hyperspectral data. Specifically, based on tensor low-rank representation (TLRR) learning, both low-rank constraints and sparsity constraints are simultaneously imposed on the coefficient tensor to capture the global and local spatial-spectral structure information of hyperspectral image (HSI), respectively. For anomaly tensor, the tensor ℓ21-norm is applied to encourage the group sparsity of anomalous pixels. Furthermore, the tensor robust principal component analysis (TRPCA) approach is utilized to construct a robust background dictionary tensor. Experimental results gained employing two real hyperspectral datasets prove the superiority of the proposed approach compared to some state-of-the-art algorithms.

Nowadays, some tensor-based hyperspectral anomaly detection (HAD) approaches are still insufficient in utilizing the spatial-spectral structure information of hyperspectral images (HSIs), resulting in the inability to isolate the background and abnormal targets well. In this letter, an enhanced tensor low-rank representation learning (ETLR) model is proposed for HAD. Specifically, the original 3-D hyperspectral image (HSI) data is firstly decomposed into a structural background component, an anomaly component and a noise component. Among them, with the help of multi-subspace learning technology, the structural background component is reformulated by the t-product of the background dictionary tensor and the corresponding coefficient tensor. Then, tensor nuclear norm (TNN) is adopted to preserve the global low-rank property of the background component in both spatial and spectral dimensions. For the abnormal component, an ℓ2,1,1-norm is designed to enhance the group sparsity of abnormal pixels. For the noise component, a tensorF-norm constraint is imposed to suppress the confusion of noise and anomalies. Meanwhile, a robust dictionary tensor that can adequately characterize the background is constructed by using tensor robust principal component analysis (TRPCA). Furthermore, to reduce the interference of redundant information on detection accuracy, the optimal clustering framework (OCF) method is utilized for band selection. Finally, extensive experiments on one simulated and three real HSI datasets confirm that our algorithm is superior than current HAD algorithms.

The ubiquity of mixed pixels in hyperspectral images makes it difficult for traditional classification techniques to determine the spatial distribution of land cover classes accurately. Subpixel mapping (SPM) technology is an effective method to solve this problem. Aiming at taking the multiple scales and the spatial features into account, an SPM method based on multi-scale and multi-feature (MSMF) is proposed, so as to effectively improve the accuracy of SPM. Firstly, the maximum linearization index method of the non-redundant complete straight-line set is designed to identify the linear distribution feature of land-cover classes. And then, different methods are applied to different spatial features and unified together finally, where the template matching iterative exchange is used for the linear distribution classes, and the multiscale spatial dependence iterative exchange method combined with area perimeter is used for the planar distribution classes. Experiments on three remote sensing images are carried out to evaluate the performance of MSMF. The results show that the proposed method can effectively improve the accuracy of SPM.

Recently, hyperspectral anomaly detection (HAD) methods based on tensor low-rank representation (TLRR) have received widespread attention. However, most of them tend to emphasize the utilization of multiple types of prior knowledge to characterize background components, while the prior information about anomaly components is limited. Additionally, the constructed background dictionary is also susceptible to noise and outliers. To address these challenges, this paper focuses on both the background and abnormal components, proposing a robust tensor low-rank sparse representation with saliency prior (RTLSR-SP) method for HAD. Specifically, for the background component described by the dictionary tensor and the corresponding coefficient tensor, tensor nuclear norm (TNN) constraint and sparsity constraint are imposed on the coefficient tensor simultaneously to capture the global and local spatial-spectral structure information of the hyperspectral image (HSI), respectively. For the anomalous component, we design a sparse saliency prior weight tensor to enhance the saliency of anomalous targets. Meanwhile, the tensor ℓF,1-norm is also integrated into the model to better separate abnormal targets from the background. Furthermore, combining tensor robust principal component analysis (TRPCA) and skinny tensor singular value decomposition (skinny t-SVD), a robust background dictionary is constructed. Finally, an efficient iterative algorithm based on the alternating direction method of multipliers (ADMM) is derived to optimize the RTLSR-SP model. Comprehensive experimental findings on one simulated dataset and six real hyperspectral datasets demonstrate the effectiveness and superiority of the proposed algorithm compared with eight state-of-the-art HAD algorithms.

Low rank and sparse representation (LRaSR)-based approaches have been widely used for hyperspectral anomaly detection (HAD) by effectively decomposing a hyperspectral data set into a low - rank component for background (BKG), a sparse component for anomalies, and a residual component for noise. This paper proposed a novel hyperspectral anomaly detection (AD) method based on data sphering (DS) and unsupervised target detection with sparse cardinality (DS-UTSSC). First of all, DS-UTS uses data sphering to remove BKG, which is characterized by 1 st and 2ndstatistics, for original data$X$. Second, potential anomaly component$\mathrm{S}_{\text{DS}-\mathrm{U}\text{TS}}$is generated via unsupervised target detection and subspace projection for the sphered data$X$. To further reduce the impact of noise on anomaly detection, sparse cardinality (SC) is incorporated to obtain$\mathrm{S}_{\text{DS}-\text{UTSSC}}$. Finally, RX-AD is implemented on$\mathrm{S}_{\mathrm{D}\text{S-UTSSC}}$to detect anomalies. The experimental results validate that DS-UTSSC is very competitive against the LRaSR-based models and AE-based method.

Hyperspectral image (HSI) change detection (CD) has gained much attention in remote sensing. However, most deep learning methods are restricted by a limited receptive field, without leveraging temporal information, and the need for many training samples. In this letter, we proposed a Transformer Encoder-based HSI CD framework called CDFormer. First, space and time encodings are added to the pixel sequence to guide transformers to exploit change information of space and time by the pixel embedding (PE) module. Second, the self-attention component of Transformer Encoder module has a global space-time receptive field to mine the correlation and interaction between bi-temporal features, enhancing the utilization of temporal dependencies. Next, the multi-head attention mechanism learns several attentions and extracts the joint weighted spatial-spectral-temporal features, which improves the feature discrimination ability of the changes. Finally, the detection result is predicted using a fully connected network. It is notable to mention that the proposed method only uses a few labeled samples to train the network. Experiments on two HSI datasets demonstrate that our proposed method can get effective performance in HSI CD.

Spatial information can play a supporting role in spectral unmixing. In this letter, we propose a dual branch autoencoder network to incorporate spatial-contextual information for spectral-spatial unmixing. The two branches leverage different architectures to efficiently extract spatial information and spectral information. In the first branch, we use fully connected layers to extract spectral information, where the neuron in each layer can capture all spectral features. In the second branch, 2-D convolution is adopted to exploit spatial features, which does not require hand-crafted assumptions compared with conventional methods. Then the extracted features are concatenated and propagated to generate the abundance and reconstruct the pixel. Moreover, to solve the drawbacks of the existing reconstruction functions, we propose a new function termed squared sine distance to improve the convergence quality of the proposed network. Experimental results reveal the effectiveness of our proposed method on both synthetic data and real-world data.

With the development of deep learning, the benchmark of hyperspectral imagery classification is constantly improving, but there are still significant challenges for hyperspectral imagery classification of few-shot scenes. This letter proposes an active-learning-based prototypical network (ALPN), which uses the prototypical network to extract representative features from a few samples. Moreover, it combines semisupervised clustering and active learning methods to select and request labels from valuable examples actively. In this way, the feature extraction ability of the network is gradually optimized. The experimental results validated that the classification accuracy and robustness of ALPN significant exceeded the comparison baselines. Furthermore, because it can be regarded as a sample selection method, ALPN can be easily combined with other models to obtain better classification results.

With the development of deep learning, hyperspectral image classification (HSIC) has improved rapidly in recent years. Unsupervised feature learning algorithms play an important role in extracting features from hyperspectral images (HSIs). This letter proposed a random-occlusion-based Bootstrap-Your-Own-Latent network (ROBYOL), combining a new augmentation method and a superior contrastive learning algorithm for feature extraction. The proposed method consists of a self-supervised learning part for feature extracting and a classifier part as the downstream task. It can be proved by the experimental results that the feature extraction ability of the network is effective in this way. Furthermore, the influence of different occlusion strategies is also studied, including changing occlusion area and occlusion value, and we proposed translucent occlusion. Our results with two well-known HSIs reveal that proper occlusion strategies can improve hyperspectral classification results effectively.

Band selection (BS) is an effective means to solve the problems of spectral redundancy and Hughes phenomenon in hyperspectral images (HSIs). However, existing BS methods fail to take into account the representativeness, redundancy, and information content of the selected bands simultaneously, and most of them lack consideration of the inherent nonlinear relationship between bands. To address these problems, we propose a novel unsupervised BS framework that can comprehensively consider band representativeness, redundancy, and information content (RRI) in this letter. The band representativeness is estimated by a three-dimensional convolutional autoencoder, which can capture the inherent nonlinear relationship between the bands and leverage the spatial information of the HSI. The redundancy and the information content of a band subset are restricted and enhanced by the correlation coefficient and the information divergence, respectively. Subsequently, RRI combines these three indicators as the subset evaluation criterion and utilizes immune clone selection algorithm to search for the desired band subset. Experimental results verify that the proposed RRI method can provide higher classification accuracy than the competitors and is robust to noisy bands.

Although remote sensing image registration has been studied for several years, achieving accurate image registration remains a challenging task due to the complicated conditions surrounding remote sensing images. To improve the accuracy and robustness of image registration, we proposed a registration method based on the global mixed structure similarity (GMSIM) measure. This measure mixes the structure similarity in both the frequency domain and the intensity domain because phase-based structure similarity in the frequency domain is sensitive to intensity contrast and spatial translation, and gray-based structure similarity in the intensity domain is efficient to structure change. Feature-based registration methods are used to generate the initial registration parameters. After that, we calculate the final registration parameters by maximizing GMSIM. Quantum-behaved particle swarm optimization (QPSO) is utilized to solve the optimal results of GMSIM due to its high efficiency. The proposed method has been evaluated on several remote sensing images differing in scale, gray, and scene and compared with three state-of-the-art registration methods. Experimental results demonstrate the high accuracy of the proposed scheme.

Despite that many state-of-the-art methods have been proposed for infrared (IR) small target detection, target detectability (TD) and background suppressibility (BS) cannot be significantly improved simultaneously, especially in complex situations. This article proposes a novel IR small target detection method named improved multimode nuclear norm joint local weighted entropy contrast (IMNN-LWEC), which represents the IR target detection task as an optimization problem for tensor decomposition of three components in the spatial–temporal domain, including background tensor, target tensor, and sparse structure tensor. First, to utilize the spatial and temporal information in an IR sequence effectively, we transform the original IR sequence into a nonoverlapping spatial–temporal patch tensor. Second, a nonconvex approximation of tensor rank called improved multimode weighted tensor nuclear norm (IMWTNN) is proposed to estimate background tensor rank, which is of benefit to separate the background component more completely from the original image. Third, based on the structure tensor theory, we introduce a new sparse prior map called LWEC via a designed image entropy operator and a new prior information filter, which can further preserve the target and suppress the background simultaneously. Besides, a novel tubewise sparse regularization term is designed to identify linear sparse structures. The Frobenius norm is used to characterize noise. Finally, to solve the proposed model, an efficient optimization scheme utilizing the alternating direction method of multipliers (ADMM) is designed to retrieve the small targets. Comprehensive experiments on five datasets witness the superior TD and BS performance of the proposed method compared with nine state-of-the-art detection methods.

Autoencoder is an efficient technique for unsupervised feature learning, which can be applied to hyperspectral unmixing. In this letter, we present an autoencoder network with adaptive abundance smoothing (AAS) to solve the challenges of previous techniques. Specifically, the proposed method uses a multilayer encoder to obtain the abundance and a single-layer decoder to reconstruct the image. The AAS algorithm tackles the outliers by exploiting the spatial-contextual information and can be adaptive for each pixel. Moreover, the softmax function is used as the encoder output function with the help of L1/2regularization to produce sparse output. Experimental results of the synthetic and real data reveal the superior performance of the proposed method against other competitors.

In recent years, low-rank representation (LRR) has attracted considerable attention in the field of hyperspectral anomaly detection. The main objective of LRR-based methods is to extract anomalies from the complex background. However, the presence of anomalies in the background dictionary can lower the detection performance. In this letter, a novel method is proposed for hyperspectral anomaly detection based on the LRR model. This method facilitates the discrimination between the anomalous targets and background by utilizing a novel dictionary and an adaptive filter based on the local outlier factor (LOF). In order to exclude the potential anomalies from the dictionary, the ranking of LOF scores for each pixel is adapted to select the potential background pixels as dictionary atoms. A filter that explores the intrinsic spatial structure is designed to enhance the differences between the anomalies and the background pixels. The experimental results that conducted on three real-world data sets demonstrate that the proposed method achieves a better performance than several state-of-the-art hyperspectral anomaly detection methods.

Due to significant geometric distortions and illumination differences, developing techniques for high precision and robust multisource remote sensing image registration poses a great challenge. This article presents an iterative image registration approach, called iterative scale-invariant feature transform (ISIFT) for remote sensing images, which extends the traditional scale-invariant feature transform (SIFT)-based registration system to a close-feedback SIFT system that includes a rectification feedback loop to update rectified parameters in an iterative manner. Its key idea uses consistent feature point sets obtained by maximum similarity to calculate new alignment parameters to rectify the current sensed image and the resulting rectified sensed image is then fed back to update and replace the current sensed image as a new sensed image to reimplement SIFT for next iteration. The same process is repeated iteratively until an automatic stopping rule is satisfied. To evaluate the performance of ISIFT, both the simulated and real images are used for experiments for the validation of ISIFT. In addition, several data sets are particularly designed to conduct a comparative study and analysis with existing state-of-the-art methods. Furthermore, experiments with different rotation are also performed to verify the adaptability of ISIFT under different rotation distortions. The experimental results demonstrate that ISIFT improves performance and produces better registration accuracy than traditional SIFT-based methods and existing state-of-the-art methods.

Hyperspectral unmixing (HU) is a crucial technique for exploiting remotely sensed hyperspectral data, which aims at estimating a set of spectral signatures, called endmembers and their corresponding proportions, called abundances. The performance of HU is often seriously degraded by various kinds of noise existing in hyperspectral images (HSIs). Most of existing robust HU methods are based on the assumption that noise or outlier only exists in one kind of formulation, e.g., band noise or pixel noise. However, in real-world applications, HSIs are unavoidably corrupted by noisy bands and noisy pixels simultaneously, which require robust HU in both the spatial dimension and spectral dimension. Meanwhile, the sparsity of abundances is an inherent property of HSIs and different regions in an HSI may possess various sparsity levels across locations. This article proposes a correntropy-based spatial-spectral robust sparsity-regularized unmixing model to achieve 2-D robustness and adaptive weighted sparsity constraint for abundances simultaneously. The updated rules of the proposed model are efficient to be implemented and carried out by a half-quadratic technique. The experimental results obtained by both synthetic and real hyperspectral data demonstrate the superiority of the proposed method compared to the state-of-the-art methods.

The orthogonal-projection-based band selection (OPBS) algorithm is one of the newly proposed band selection methods. In this letter, we present a nonlinear version of the OPBS method, which is denoted as the Kernel-OPBS method. The OPBS method selects the desired bands one by one, and in each round of lookup, it chooses the band that has the maximum distance to the hyperplane spanned by the currently selected bands. Extending this algorithm to a feature space associated with the original input space through a certain nonlinear mapping function can provide a nonlinear version of the OPBS algorithm. Although it is basically intractable to compute the mapped bands due to the high dimensionality of the feature space produced by the nonlinear mapping function, the selection criterion of the Kernel-OPBS method is actually related to only the inner products of the mapped bands; thus, the kernel function can be applied and it is unnecessary to define the nonlinear mapping function. Experimental results on different data sets demonstrate that the selected bands obtained by the Kernel-OPBS method can achieve higher pixel classification performances than that by the OPBS method.

This letter presents a superpixel-based dual window RX (SPDWRX) anomaly detection (AD) algorithm that uses superpixel segmentation (SPS) to adaptively determine the dual window for local RX (LRX) detection, rather than using a fixed dual window. The main premise of SPDWRX is to first divide the hyperspectral image into multiple superpixels and then extend the minimum bounding rectangle to determine the background of each superpixel. Finally, LRX AD is conducted on each pixel in the same superpixel using the same background. Furthermore, a fine SPS method is proposed based on the entropy rate superpixel to quickly obtain uniform superpixels. The experimental results show that the proposed SPDWRX method can significantly improve the detection speed and slightly improve the detection performance, and the modified SPS can further improve the detection performance of SPDWRX.

Infrared small target detection is one of the most important parts of infrared search and tracking (IRST) system. Generally, the small and dim target is of low signal-to-noise ratio and buried in the complicated background and heavy noise, which makes it extremely difficult to be detected with low false alarm rates. To solve this problem, we propose a small target detection method based on multiscale local contrast measure. Different from conventional methods, we novelly measure the local contrast from two aspects: local dissimilarity and local brightness difference. First, we present a new dissimilarity measure called the local energy factor (LEF) to describe the dissimilarity between the small targets and their surrounding backgrounds. Second, the feature of the brightness difference between the small targets and the backgrounds is utilized. Afterward, the local contrast is measured by taking both features of the above into account. Finally, an adaptive segmentation method is applied to extract the small targets from the backgrounds. Extensive experiments on real test data set demonstrate that our approach outperforms the state-of-the-art approaches.

In the field of deep learning, finetuning the pretrained networks to get a good classifier is a common way of transfer learning. Unlike the traditional way, we insert deformable convolutional layers into the pretrained networks, and finetune the new networks. As a result, we find it performs as well as the normal one in classification, and when we construct a plane detection pipeline based on the two classifiers respectively, the one with deformable convolution shows a better result than the other.

Spectral unmixing is a fundamental issue that needs to be addressed in the application of hyperspectral images. Due to the complex imaging conditions in remote sensing, it is common for the same object to have different spectral signatures. In this paper, we present a novel endmember bundle extraction method based on pixel purity index and superpixel segmentation to deal with this problem, where each material is represented by a set of similar endmember spectra. This method improves the accuracy of endmember extraction and focuses on the removal of redundant endmembers, leading to less spectral unmixing error than existing endmember bundle extraction algorithms. The experimental results on both synthetic dataset and real dataset demonstrate that the proposed method performs effectively in extracting variable endmember sets.

Nonnegative matrix factorization (NMF) and its various robust extensions have been widely applied to hyperspectral unmixing. Most existing robust NMF methods consider that noises only exist in one kind of formulation. However, hyperspectral images (HSI) are unavoidably corrupted by noisy bands and noisy pixels simultaneously in the real application s. This paper presents a robust NMF using ℓ1,2norm and further proposes a two-dimensional robust NMF model by incorporating ℓ2,1norm and ℓ1,2norm, which is robust to noises in both spatial dimension and spectral dimension simultaneously. In addition, the Huber's M-estimator is integrated into the model to achieve better assignations of weights for each pixel and band with various noise intensities, which avoids the singularity problem and effectively improves the unmixing performance. The elegant updating rules of the proposed model are also efficiently learnt and provided. Experiments are conducted on both synthetic and real hyperspectral data sets. The experimental results demonstrate the effectiveness of the proposed methods in unmixing performance.

This paper proposes a new unsupervised band selection method for hyperspectral imagery anomaly detection. Main background subset is identified using global RX detector. An effective criterion of band subset selection is designed based on minimum background variations using differential data of the main background subset. The particle swarm optimization algorithm is used as the search strategy to find the best solution for band selection with the proposed criteria. The proposed method is evaluated by experiments of hyperspectral anomaly detection, and the results show that the performance of anomaly detection is significantly improved after band selection.

Hyperspectral unmixing (HU) is a crucial technique for exploiting remotely sensed hyperspectral data, which aims to estimate a set of spectral signatures, called endmembers and their corresponding proportions, called abundances. Nonnegative matrix factorization (NMF) and its various robust extensions have been widely applied to HU. Most existing robust NMF methods consider that noises only exist in one kind of formulation. However, the hyperspectral images (HSIs) are unavoidably corrupted by noisy bands and noisy pixels simultaneously in the real applications. This paper proposes a novel spectral-spatial robust NMF model by incorporating 12,1 norm and 11,2 norm, which achieves robustness to band noise and pixel noise simultaneously. The Huber's M-estimator is integrated into the proposed model to achieve better assignations of weights for each pixel and band with various noise intensities, which avoids the singularity problem and effectively improves the unmixing performance. The elegant updating rules of the proposed spectral-spatial robust model are also efficiently learned and provided. Experiments are conducted on both synthetic and real hyperspectral data sets. The experimental results demonstrate the effectiveness of the proposed methods in unmixing performance.

Band selection (BS) tries to find a few useful bands to represent the whole hyperspectral image cube. This letter proposes a novel unsupervised BS method based on the band correlation analysis (BCA). The BCA method tries to find a subset of bands that can well represent the whole image data set. To avoid the exhaustive search, the BCA method iteratively adds the band with the good representative ability and low redundancy into the selected band set, until the sufficient quantity of bands has been obtained. The redundancy and the representative ability of one band are computed by its correlation with the currently selected bands and the remaining unselected bands, respectively. Through constructing a correlation matrix of total bands, the BCA method can find the bands that with large amounts of information and low redundancy, which ensures that the selected bands are useful for the further applications like pixels classification. Experimental results on three different data sets demonstrate that the proposed method is very effective and can achieve the best performance among the competitors.

Kernel nonnegative matrix factorization (KNMF) is an extension of NMF designed to capture nonlinear dependence features in data matrix through kernel functions. In KNMF, the size of the kernel matrices is closely associated with the input data matrix, of which the calculation consumes a large amount of memory and computing resource. When applied on large-scale hyperspectral data, KNMF often meets the bottleneck of memory and may cause the overflow of memory. And when dealing with dynamically acquired data, KNMF requires recomputation of the whole data set when newly acquired data arrived, which produces huge memory and computing resource requirements. To reduce the usage of memory and improve the computational efficiency when applying KNMF on large scale and dynamic hyperspectral data, we extend KNMF by introducing partition matrix theory and considering the relationships among dividing blocks. The decomposition results of hyperspectral data are derived from much smaller scale matrices containing the formerly achieved results and the newly data blocks incrementally. In this paper, we propose an incremental KNMF (IKNMF) to reduce the computing requirements for large-scale data in hyperspectral unmixing. An improved IKNMF (IIKNMF) is also proposed to further improve the abundance results of IKNMF. Experiments are conducted on both synthetic and real hyperspectral data sets. The experimental results demonstrate that the proposed methods can effectively save memory resources without degrading the unmixing performance and the proposed IIKNMF can achieve better abundance results than IKNMF and KNMF.

Band selection (BS) is a special case of the feature selection problem, and it tries to remove redundant bands and select a few informative and distinctive bands to represent the whole image cube. The maximum ellipsoid volume (MEV) method regards the band subset with the maximum volume as the optimal band combination. However, the MEV method cannot be directly applied for hyperspectral imagery due to the high dimensionality of the data sets. Therefore, we first combine MEV with the sequential forward search (SFS) and propose a new unsupervised BS method called MEV-SFS. Furthermore, a subtle relationship between the ellipsoid volume of the band set and the orthogonal projections (OPs) of the candidate bands is observed. Based on this relationship, we propose another equivalent method, namely, the OP-based BS (OPBS) method. OPBS is the fast version of MEV-SFS, and it has a better computational efficiency and the potential to determine the number of bands to be selected. We specifically explain the rationality of the MEV-based methods (MEV-SFS and OPBS) and illustrate their theoretical significance and physical meaning from different aspects. Theoretical analysis also demonstrates that OPBS can be regarded as a model or framework for BS, and thus, we further propose a third novel BS method named the OPBS-information divergence (OPBS-ID) method, which is a variant of OPBS. OPBS-ID can achieve a better classification performance than OPBS in many cases. Experimental results on different hyperspectral data sets demonstrate that the proposed methods have high computational efficiency, and the selected bands can achieve satisfactory classification performances.

This paper proposed a novel local feature descriptor for multisource remote sensing image matching that is robust to significant geometric and illumination differences. In the proposed registration method, traditional SIFT algorithm is applied for local feature extraction and a novel descriptor, named local order pattern based self-similarity descriptor, LOPSS descriptor, is constructed for each extracted feature point. Then, a matching process followed by a reliable outlier removal procedure is implemented for feature matching and mismatch elimination. Finally, registration parameters are estimated by least square method in the affine transformation. The proposed method is applied for matching multisource remote sensing image pairs and the results verify its robustness and discriminability.

Hyperspectral unmixing aims to estimate a set of endmembers and their corresponding percentages in pixels. NMF and its extensions with various constraints have been widely applied to hyperspectral unmixing. L1/2 regularizer and L2 regularizer can be added into NMF to enforce sparseness and smoothness respectively. In practice, an rigion in hyperspectral image may possesses different sparse level across locations. It remains a problem how to impose constraints accordingly when the level of sparse varies. We propose a novel nonnegative matrix factorization with data-guided constraints (DGC-NMF). The DGC-NMF assigns sparseness or smoothness constraints on abundance of each pixel individually according to their mixed level. Experiments on the synthetic data validate the proposed algorithm.

This paper proposes a novel Bayesian Lasso model with spatially related sparsity for hyperspectral linear unmixing. Based on the sparsity hypothesis and the spatial correlation between pixels, we introduce Lasso penalty and spatially constrained prior distributions to the original Beyesian model. This prior information takes into consideration that different regions in the hyperspectral possess various sparse level and the pixels in a same region tends to share a similar mixed level. We assign relatively slighter constraint to the transition areas which generally have higher mixed levels, by measuring spatial correlation adapting to the prior of abundance vector. Empirical experiments show attractive results of the proposed method via extensive simulation studies and comparisons with other algorithms.

The minimization problem of reconstruction error over large hyperspectral image data is one of the most important problems in unsupervised hyperspectral unmixing. A variety of algorithms based on nonnegative matrix factorization (NMF) have been proposed in the literature to solve this minimization problem. One popular optimization method for NMF is the projected gradient descent (PGD). However, as the algorithm must compute the full gradient on the entire dataset at every iteration, the PGD suffers from high computational cost in the large-scale real hyperspectral image. In this paper, we try to alleviate this problem by introducing a mini-batch gradient descent based algorithm, which has been widely used in large-scale machine learning. In our method, the endmember can be updated pixel set by pixel set while abundance can be updated band set by band set. Thus, the computational cost is lowered to a certain extent. The performance of the proposed algorithm is quantified in the experiment on synthetic and real data.

High-precision and robust matching of multi-source remote sensing image matching is not easy to achieve because of non-linear intensity differences and significant geometric distortions. A new registration method is proposed by integrating the scale-invariant feature transform (SIFT) and optimization of local self-similarity mutual information (LSS_MI). This method consists of two main steps. In the first step, SIFT approach with a reliable outlier removal procedure is implemented. By repeatedly fine turning several selected matched feature point coordination, a series of registration parameters are estimated by least square method and used to construct initial particle swarms. Then, a local self-similarity descriptor (LSS) is computed for pre-matching image pairs and the optimal match parameters are obtained by optimizing LSS_MI based on QPSO. The experimental results verify that the LSS-MI is more robust and accurate than regional mutual information in multi-source remote sensing image.

In this paper, we proposed an incremental kernel non-negative matrix factorization (IKNMF) to reduce the computing scale in hyperspectral unmixing. Kernel non-negative matrix factorization (KNMF) is an extended non-negative matrix factorization (NMF) able to capture nonlinear dependency features in data matrix through kernel functions. In KNMF algorithm, the size of kernel matrices is closely associated with the input data scale. To reduce calculation and storage of large matrices, we extend KNMF by introducing partition matrix theory. The decomposition results of data matrices are derived from smaller scale matrices incrementally. Experiments are conducted on synthetic hyperspectral images with multiple sizes, and the experimental results show that the proposed algorithm have effect in saving calculation and memory resource without degrading the unmixing performance.

Most nonlinear unmixing algorithms are based on the nonlinear mixing models with different forms. This paper focuses on the well-known generalized bilinear model (GBM). Though the GBM has shown interesting and promising for nonlinear unmixing, currently almost all the GBM-based unmixing algorithms are supervised. That is, the endmembers must be assumed known in advance. This paper develops an unsupervised nonlinear unmixing method based on the GBM, which can obtain the endmember, abundances and nonlinearity coefficients simultaneously. In the proposed method, the projected-gradient (PG) algorithm are utilized to alternately solve two nonnegative matrix factorization problems. The former updates the endmembers while the latter updates the abundances as well as the nonlinearity coefficients. Experimental results show that the proposed algorithm provide good performance in term of both endmember estimation and abundances estimation comparing with other state-of-the-art algorithms.

To select a minimal and effective subset from a mass of bands is one key issue in hyperspectral image processing. This paper proposed a novel band selection approach using mutual information and K-L divergence. Mutual information (MI) is usually used to measure the statistical dependence between two random variables and can be used to evaluate the relativity of each band. Firstly all bands are grouped into a number of subsets using mutual information. Then, retain only specified number of bands that has maximum information amount which defined by K-L divergence in every subset by removing the others. At last, the retained bands consist of the final bands collection. Experimental results of real hyperspectral dataset show that the proposed algorithm reduces the dimensionality of the data significantly, as well as keeps a better precision.

Endmember extraction is a key step in the hyperspectral image analysis process. The kernel new simplex growing algorithm (KNSGA), recently developed as a nonlinear alternative to the simplex growing algorithm (SGA), has proven a promising endmember extraction technique. However, KNSGA still suffers from two issues limiting its application. First, its random initialization leads to inconsistency in final results; second, excessive computation is caused by the iterations of a simplex volume calculation. To solve the first issue, the spatial pixel purity index (SPPI) method is used in this study to extract the first endmember, eliminating the initialization dependence. A novel approach tackles the second issue by initially using a modified Cholesky factorization to decompose the volume matrix into triangular matrices, in order to avoid directly computing the determinant tautologically in the simplex volume formula. Theoretical analysis and experiments on both simulated and real spectral data demonstrate that the proposed algorithm significantly reduces computational complexity, and runs faster than the original algorithm.

Nonlinear unmixing, which has attracted considerable interest from researchers and developers, has been successfully applied in many real-world hyperspectral imaging scenarios. Hopfield neural network (HNN) machine learning has already proven successful in solving the linear mixture model; this study utilized an HNN machine learning approach to solve the generalized bilinear model (GBM) optimization problem. Two HNNs were constructed in a successive manner to solve respective seminonnegative matrix factorization problems intended for abundance and nonlinear coefficient estimation. In the proposed HNN-based GBM unmixing method, both HNNs evolve to stable states after a number of iterations to obtain unmixing results related to the states of neurons. In experiments on synthetic data, the proposed method showed more efficient performance in regard to abundance estimation accuracy than other GBM optimization algorithms, especially when given reliable endmember spectra. The proposed method was also applied to real hyperspectral data and still demonstrated notable advantages despite the obvious increase in unmixing difficulty.

One key issue encountered in endmember extraction is to determine the number of endmembers, p, required to be extracted. Virtual dimensionality (VD) has been widely used for this purpose. However, VD was originally developed and defined as the number of spectrally distinct signatures which are not necessarily pure signatures. So, on some occasions the VD estimated value for p may not be accurate to be used for the number of endmembers. This paper develops an endmember-specified VD (ES-VD) which makes use of data sample vectors generated by a specific endemember finding algorithm (EFA) as target signal sources and then determine if these signal sources are indeed true endmember by a binary composite hypothesis testing to determine endmembers. As a result, ES-VD varies with different target signal sources produced by EFAs. Most importantly, ES-VD not only determines the value of VD and in the mean time it also finds desired endmembers.

Nonlinear spectral mixture analysis is important when the light suffers multiple interactions among distinct materials. Few attempts have been conducted to incorporate spatial information to improve the performance of nonlinear unmixing algorithms. In this letter, local windows are adopted in the preliminary classification map to search the relevant endmembers for each pixel. Virtual endmembers, resulting from the relevant endmembers, represent the multiple-scattering effects in each pixel, and the corresponding abundances are estimated based on a modified bilinear model. Experiments on simulated and real hyperspectral images demonstrate that the proposed method provides a competitive or even better performance over some existing algorithms.

Endmember extraction is an important and challenging step to solve the spectral unmixing problem. Most existing endmember extraction algorithms (EEAs) usually find image pixels as endmembers assuming the presence of pure pixels in an image scene or generate virtual endmembers without pure-pixel assumption. When some prevalent materials have pure-pixel representation and pure pixels of other less prevalent materials are absent in the image, it would be more appropriate to extract the endmembers of both prevalent and less prevalent materials, respectively. Therefore, a novel two-stage EEA is presented in this paper. In the first stage, conventional pure-pixel-based EEAs are applied to generate a candidate pixel set, and then spatial information of the candidate pixels is exploited to determine the endmembers of prevalent materials. In the second stage, given known endmembers of prevalent materials, a modified algorithm based on nonnegative matrix factorization is performed to generate the endmembers of less prevalent materials. The validity of the proposed algorithm is demonstrated by experiments based on synthetic mixtures and a real image scene.