In this paper, we propose a novel unsupervised shape matching framework based on probabilistic deformation consistency in the spectral domain, termed as PDCMatch. Axiomatic optimization methods suffer from expensive geodesic distance calculations and vulnerability to local optima, and learning-based methods typically lack geometric consistency in pointwise correspondences. To overcome both limitations, we develop a non-Euclidean probabilistic deformation model that jointly estimates the underlying deformation and the correspondence probability via a linear Expectation-Maximization procedure. Building on this formulation, we further design a task-specific deformation loss that explicitly encourages geometric smoothness and structural consistency in an unsupervised manner. This tailored loss function plays a central role in improving the matching performance across challenging scenarios. Extensive experiments on public benchmarks involving near-isometric shapes, anisotropic meshing, cross-dataset generalization, topological noise, and non-isometric shapes demonstrate that our method consistently outperforms state-of-the-art methods, highlighting both its effectiveness and generalizability.

Establishing point-to-point correspondences across multiple 3D shapes is a fundamental problem in computer vision and graphics. In this paper, we introduce DcMatch, a novel unsupervised learning framework for non-rigid multi-shape matching. Unlike existing methods that learn a canonical embedding from a single shape, our approach leverages a shape graph attention network to capture the underlying manifold structure of the entire shape collection. This enables the construction of a more expressive and robust shared latent space, leading to more consistent shape-to-universe correspondences via a universe predictor. Simultaneously, we represent these correspondences in both the spatial and spectral domains and enforce their alignment in the shared universe space through a novel cycle consistency loss. This dual-level consistency fosters more accurate and coherent mappings. Extensive experiments on several challenging benchmarks demonstrate that our method consistently outperforms previous state-of-the-art approaches across diverse multi-shape matching scenarios.

Two-view correspondence learning aims to discern true and false correspondences between image pairs by recognizing their underlying different information. Previous methods either treat the information equally or require the explicit storage of the entire context, tending to be laborious in real-world scenarios. Inspired by Mamba's inherent selectivity, we propose CorrMamba, a Correspondence filter leveraging Mamba's ability to selectively mine information from true correspondences while mitigating interference from false ones, thus achieving adaptive focus at a lower cost. To prevent Mamba from being potentially impacted by unordered keypoints that obscured its ability to mine spatial information, we customize a causal sequential learning approach based on the Gumbel-Softmax technique to establish causal dependencies between features in a fully autonomous and differentiable manner. Additionally, a local-context enhancement module is designed to capture critical contextual cues essential for correspondence pruning, complementing the core framework. Extensive experiments on relative pose estimation, visual localization, and analysis demonstrate that CorrMamba achieves state-of-the-art performance. Notably, in outdoor relative pose estimation, our method surpasses the previous SOTA by 2.58 absolute percentage points in AUC@20°, highlighting its practical superiority. Our code is publicly available at https://github.com/ShineFox/CorrMamba.

The differences in imaging devices cause multimodal images to have modal differences and geometric distortions, complicating the matching task. Deep learning-based matching methods struggle with multimodal images due to the lack of large annotated multimodal datasets. To address these challenges, we propose XCP-Match based on cross-modality completion pre-training. XCP-Match has two phases. (1) Self-supervised cross-modality completion pre-training based on real multimodal image dataset. We develop a novel pre-training model to learn cross-modal semantic features. The pre-training uses masked image modeling method for cross-modality completion, and introduces an attention-weighted contrastive loss to emphasize matching in overlapping areas. (2) Supervised fine-tuning for multimodal image matching based on the augmented MegaDepth dataset. XCP-Match constructs a complete matching framework to overcome geometric distortions and achieve precise matching. Two-phase training encourages the model to learn deep cross-modal semantic information, improving adaptation to modal differences without needing large annotated datasets. Experiments demonstrate that XCP-Match outperforms existing algorithms on public datasets.

Recent advances in deep learning-based methods have led to significant progress in the hyperspectral super-resolution (SR). However, the scarcity and the high dimension of data have hindered further development since deep models require sufficient data to learn stable patterns. Moreover, the huge domain differences between hyperspectral image (HSI) datasets pose a significant challenge in generalizability. To address these problems, we present a general hyperspectral SR framework via meta-transfer learning (MTL). We randomly sample various spectral ranges for SR tasks during MTL, allowing the model to accumulate diverse task experiences. Additionally, we implement a task schedule to gradually expand the number of bands, bridging the significant domain differences between datasets. By leveraging multiple datasets, we are able to achieve better performance and greater generalizability, making it applicable under various circumstances. Meanwhile, as a general framework, our scheme can be applied to existing methods to obtain performance improvements. In addition, we design an advanced network architecture based on the multifusion features to further improve the performance. Experiments demonstrate that our method not only achieves superior performance in both qualitative and quantitative terms but also can adapt robustly to a new and difficult sample, where few epochs can yield quite considerable results.

Existing deep pan-sharpening methods lack the learning of complementary information between PAN and MS modalities in the intermediate layers, and exhibit low interpretability due to their black-box designs. To this end, an interpretable deep unfolded network with intrinsic supervision for pan-sharpening is proposed. Building upon the observation degradation process, it formulates the pan-sharpening task as a variational model minimization with spatial consistency prior and spectral projection prior. The former prior requires a joint component decomposition of PAN and MS images to extract intrinsic features. By being supervised in the intermediate layers, it can selectively provide high-frequency information for spatial enhancement. The latter prior constrains the intensity correlation between MS and PAN images derived from physical observations, so as to improve spectral fidelity. To further enhance the transparency of network design, we develop an iterative solution algorithm following the half-quadratic splitting to unfold the deep model. It rigorously adheres to the variational model, significantly enhancing the interpretability behind network design and efficiently alternating the optimization of the network. Extensive experiments demonstrate the advantages of our method compared to state-of-the-arts, showcasing its remarkable generalization capability to real-world scenes. Our code is publicly available at https://github.com/Baixuzx7/DISPNet.

In the realm of AI, data serves as a pivotal resource. Real-world hyperspectral images (HSIs), bearing wide spectral characteristics, are particularly valuable. However, the acquisition of HSIs is always costly and time-intensive, resulting in a severe data-thirsty issue in HSI research and applications. Current solutions have not been able to generate a sufficient volume of diverse and reliable synthetic HSIs. To this end, our study formulates a novel, generalized paradigm for HSI synthesis, i.e., unmixing before fusion, that initiates with unmixing across multi-source data and follows by fusion-based synthesis. By integrating unmixing, this work maps unpaired HSI and RGB data to a low-dimensional abundance space, greatly alleviating the difficulty of generating high-dimensional samples. Moreover, incorporating abundances inferred from unpaired RGB images into generative models allows for cost-effective supplementation of various realistic spatial distributions in abundance synthesis. Our proposed paradigm can be instrumental with a series of deep generative models, filling a significant gap in the field and enabling the generation of vast high-quality HSI samples for large-scale downstream tasks. Extension experiments on downstream tasks demonstrate the effectiveness of synthesized HSIs. The code is available at HSI-Synthesis.github.io.

With the lack of sufficient prior information, unsupervised hyperspectral unmixing (HU) has been a preprocessing step in the hyperspectral image (HSI) processing pipeline, which can provide the types of material and corresponding abundance information of HSI, to further provide assistance for downstream higher level semantic tasks to overcome the limitation caused by mixed pixels. However, the unmixing results obtained by current unsupervised HU methods are unstable and unprecise under the guidance of the least reconstruction error (RE), which have no consistency with the performance of high-level tasks. To solve this problem, this article takes the hyperspectral anomaly detection (HAD) as an entry point and proposes a novel algorithm based on deep clustering which can jointly perform HU and HAD in an end-to-end manner. A mutual feedback mechanism is formed between the upstream HU process and the downstream HAD process, and through joint optimization, both two tasks can achieve relatively good performances. However, the low dimensional abundance has a limited representation, which may lead to the increase of false alarm rate. To overcome this limitation, the principal components (PCs) of HSI are fused with the abundance to enhance the representation ability. Moreover, we use the reweighted reconstruction loss strategy to enhance the role of anomalies in the HU process. Experiments performed on several real datasets verify the rationality and superiority of the proposed UADNet algorithm.

The scarcity of hyperspectral images (HSIs) hinders the development of processing methods and downstream applications. HSI synthesis, which aims to generate realistic samples from the existing datasets, is undoubtedly a prospective and economical solution for the HSI data shortage problem. Inspired by the impressive performance of the diffusion model (DM) in image synthesis tasks, this article initiatively proposes an unmixing diffusion (UnmixDiff) model for high-quality HSI generation. The method starts with training an unmixing network to learn the distribution characteristic of objects (abundance). By incorporating the unmixing autoencoder into the DM, the UnmixDiff transforms the HSI generation into the abundance domain, which maintains the consistency of the generated spectral profile, reduces the computational complexity, and introduces a clear physical interpretation into the hyperspectral image synthesis tasks. After that, we construct a diffusion generation model in abundance space to generate realistic abundance maps. Instead of synthesizing original hyperspectral images, the proposed UnmixDiff synthesizes abundance maps to simulate objects’ distribution rather than superficial textures. In this way, realistic HSI samples are generated by mixing the synthesized abundance with the scene end-members. With the comparative experiments, the proposed method achieves state-of-the-art performance in HSI synthesis tasks, effectively alleviating the HSI data scarcity and supporting widespread HSI applications. The code is available athttps://github.com/yuyang95/UnmixingDM.

Hyperspectral unmixing (HU) based on linear mixing model (LMM) has received much attention in the remote sensing community over the past decades. However, many HU algorithms are based on the fixed endmember and do not fully exploit the inherent spatial characteristics of the hyperspectral image (HSI). In this letter, a new HU algorithm named GLRl1-GMM is proposed to solve these problems. We use Gaussian mixture model (GMM) to represent the endmember variability. Then, considering the local smoothness in the abundance map, a graph Laplacian regularization based onl1-norm (GLRl1) is embedded in the prior of abundance. Under the Bayesian framework, the objective density function leads to a maximum a posterior (MAP) problem, which can be solved by a generalized expectation-maximization (GEM) algotithm. Experiments on two real datasets demonstrates the effectiveness of proposed algorithm compared with other state-of-the-art methods. The source code is available at https://github.com/lwdinwhu/GMM-GLRl1.

Limited by the imaging paradigm, stripes is pervasive in remote sensing scenes, and its intensity, density, and periodicity differ dramatically among different imaging systems. Worse, it always co-exists with random noises caused by unstable imaging condition. However, current destriping methods are victim to undue ideal assumptions and fail to accurately eliminate stripes against diverse practical degradation, yielding excessive or inadequate destriping results. This study proposes a progressive hyperspectral destriping method with an adaptive frequency focus for accurate destriping and delicate restoration. Specifically, a hierarchical decomposition and reconstruction framework based on progressive wavelet learning encodes the degraded input to the frequency domain with smaller scales, easing the difficulty of restoration. Then, to avoid excessive or insufficient destriping, we devote specific efforts to finely separating noise and preserving details in the high-frequency domain. First, we devise a gradient-aware frequency attention block based on the prominent unidirectional pattern of stripes, empowering to adaptively assign weights according to their sensitivity to the spatial gradient. Second, we design a focal high-frequency loss item that is dynamically scaled according to feature distance in the high-frequency domain, profiting in identifying and preserving details. Extensive experiments conducted on data with synthetic stripes and realistic satellite scenes validate the superiority of the proposed method over the current state-of-the-art methods. The code is available at https://github.com/EtPan/PHID.

Multispectral image (MSI) and hyperspectral image (HSI) fusion can combine the best of both worlds to produce images with both high spatial and spectral resolution. In this paper, we have designed a network for fusing MSIs and HSIs, called DSPNet. On the one hand, in order to ensure the accuracy of the spectral dimension, i.e. spectral fidelity, we designed the spectral pyramid (SpePy) module and the multiscale spectral information fusion (MLSIF) module. The former extracts the multiscale local spectral information that captures the subtle spectral details and variations between different spectra. The latter establishes long-range dependency in the spectral dimension through the spectral-wise multi-head hybrid-attention (S-MHA) mechanism, thus enabling the network to focus on the local spectral information needed to recover the spectral details. On the other hand, to address the spatial information of MSIs, we designed the spatial pyramid (SpaPy) module. The SpaPy module can extract the non-local spatial information of MSIs at different scales, which enables the network to adapt to different remote-sensing scenes. Experiments performed on simulated and real data demonstrate the superiority of our method over the state-of-the-art methods both qualitatively and quantitatively.

For robust feature matching, a popular and particularly effective method is to recover smooth functions from the data to differentiate the true correspondences (inliers) from false correspondences (outliers). In the existing works, the well-established regularization theory has been extensively studied and exploited to estimate the functions while controlling its complexity to enforce the smoothness constraint, which has shown prominent advantages in this task. However, despite the theoretical optimality properties, the high complexities in both time and space are induced and become the main obstacle of their application. In this article, we propose a novel method for multivariate regression and point matching, which exploits the sparsity structure of smooth functions. Specifically, we use compact Fourier bases for constructing the function, which inherently allows a coarse-to-fine representation. The smoothness constraint can be explicitly imposed by adopting a few low-frequency bases for representation, resulting in reduced computational complexities of the induced multivariate regression algorithm. To cope with potential gross outliers, we formulate the learning problem into a Bayesian framework with latent variables indicating the inliers and outliers and a mixture model accounting for the distribution of data, where a fast expectation-maximization solution can be derived. Extensive experiments are conducted on synthetic data and real-world image matching, and point set registration datasets, which demonstrates the advantages of our method against the current state-of-the-art methods in terms of both scalability and robustness.

Spectral unmixing (SU), which refers to extracting basic features (i.e., endmembers) at the subpixel level and calculating the corresponding proportion (i.e., abundances), has become a major preprocessing technique for the hyperspectral image analysis. Since the unmixing procedure can be explained as finding a set of low-dimensional representations that reconstruct the data with their corresponding bases, autoencoders (AEs) have been effectively designed to address unsupervised SU problems. However, their ability to exploit the prior properties remains limited, and noise and initialization conditions will greatly affect the performance of unmixing. In this article, we propose a novel technique network for unsupervised unmixing which is based on the adversarial AE, termed as adversarial autoencoder network (AAENet), to address the above problems. First, the image to be unmixed is assumed to be partitioned into homogeneous regions. Then, considering the spatial correlation between local pixels, the pixels in the same region are assumed to share the same statistical properties (means and covariances) and abundance can be modeled to follow an appropriate prior distribution. Then the adversarial training procedure is adapted to transfer the spatial information into the network. By matching the aggregated posterior of the abundance with a certain prior distribution to correct the weight of unmixing, the proposed AAENet exhibits a more accurate and interpretable unmixing performance. Compared with the traditional AE method, our approach can greatly enhance the performance and robustness of the model by using the adversarial procedure and adding the abundance prior to the framework. The experiments on both the simulated and real hyperspectral data demonstrate that the proposed algorithm can outperform the other state-of-the-art methods.

In this paper, we explore a new type of extrinsic method to directly align two geometric shapes with point-to-point correspondences in ambient space by recovering a deformation, which allows more continuous and smooth maps to be obtained. Specifically, the classic coherent point drift is revisited and generalizations have been proposed. First, by observing that the deformation model is essentially defined with respect to Euclidean space, we generalize the kernel method to non-Euclidean domains. This generally leads to better results for processing shapes, which are known as two-dimensional manifolds. Second, a generalized probabilistic model is proposed to address the sensibility of coherent point drift method to local optima. Instead of directly optimizing over the objective of coherent point drift, the new model allows to focus on a group of most confident ones, thus improves the robustness of the registration system. Experiments are conducted on multiple public datasets with comparison to state-of-the-art competitors, demonstrating the superiority of our method which is both flexible and efficient to improve the matching accuracy due to our extrinsic alignment objective in ambient space.

Hyperspectral images (HSIs) with both spatial and spectral information have found broad applications. Since most cameras have narrow viewing angle, generating panoramic images is essential to show a large-range view of the environment. So far, there are few studies on HSI stitching, and the stitching result still suffers from some problems, such as blurring and ghosting, geometric misalignment, visible seam, and spectral distortion. Hence, to address the above disadvantages, we propose a novel HSI stitching strategy using optimal seamline detection approach in this letter. First, we use a fast and robust seam estimation method to determine the seamline in each single band of HSI. This method works in RGB images, and we have modified it to be used in a single-band gray-scale image of HSI. Then, to guarantee the integrity of spatial and spectral information of hundreds of bands of HSI, we propose to apply the structural similarity (SSIM) index to select the optimal one among all band candidate seamlines and use the selected optimal seamline to stitch all the remaining bands. The experimental results demonstrate that our proposed approach outperforms traditional HSI stitching approach in both spatial and spectral performances.

Anomaly detection of hyperspectral data has been gaining particular attention for its ability in detecting targets in an unsupervised manner. Autoencoder (AE), together with its variants can not only extract intrinsic features automatically but also detect anomalies that differ dramatically from others. Many AE-driven algorithms are, thus, proposed for anomaly detection in hyperspectral imagery (HSI), but they suffer from two problems: 1) when there exist anomalies in the training set, AE can generalize so well that it can also learn the abnormal patterns well, thereby reducing the ability to distinguish anomalies from the background and 2) geometric structure among samples are lost in latent space of AE, which is vital in hyperspectral anomaly detection. To tackle these problems, we propose a robust anomaly detector based on the AE framework, named robust graph AE (RGAE) detector, in this article. To be specific, we propose a robust AE framework with$\ell _{2,1}$-norm that is robust to noise and anomalies during training. Meanwhile, we embed a superpixel segmentation-based graph regularization term (SuperGraph) into AE. This strategy can preserve the geometric structure and the local spatial consistency of HSI simultaneously and also effectively reduce the searching space and execution time for each pixel. Extensive experiments are conducted on five datasets, and the results demonstrate that our method has a better detection performance, after comparing with other state-of-the-art hyperspectral anomaly detectors.

Spectral unmixing is a major technique for the further development of hyperspectral analysis. It aims to determine the corresponding proportion (fractional abundance) of the basic spectral signatures (endmembers) blindly at the subpixel level. Recently, the learning-based method has received much attention in hyperspectral unmixing, and autoencoders have been effectively designed to solve the unsupervised scenarios of unmixing. However, their ability to extract physically meaningful endmembers remains limited, and the performance has not been satisfactory. In this article, we propose a novel two-stream network, termed TANet, to address the above problems. The network consists of a two-stream architecture. First, superpixel segmentation is adopted as preprocessing to extract the endmember bundles from the image. Then, the first stream learns a mapping from the pseudopure pixels to their corresponding abundances. The second stream is conducting the same untied-weighted autoencoder to minimize reconstruction errors from the original pixel data. By learning from the pure or nearly pure candidate pixels to correct the weights of unmixing, the proposed TANet exhibits a more accurate and interpretable unmixing performance. Extensive experiments on both synthetic and real hyperspectral data demonstrate that the proposed TANet can outperform the other state-of-the-art approaches.

This article presents a novel end-to-end model based on encoder–decoder architecture for hyperspectral image (HSI) denoising, named spatial-spectral quasi-attention recurrent network, denoted as SQAD. The central goal of this work is to incorporate the intrinsic properties of HSI noise to construct a practical feature extraction module while maintaining high-quality spatial and spectral information. Accordingly, we first design a spatial-spectral quasi-recurrent attention unit (QARU) to address that issue. QARU is the basic building block in our model, consisting of spatial component and spectral component, and each of them involves a two-step calculation. Remarkably, the quasi-recurrent pooling function in the spectral component could explore the relevance of spatial features in the spectral domain. The spectral attention calculation could strengthen the correlation between adjacent spectra and provide the intrinsic properties of HSI noise distribution in the spectral dimension. Apart from this, we also design a unique skip connection consisting of channelwise concatenation and transition block in our model to convey the detailed information and promote the fusion of the low-level features with the high-level ones. Such a design helps maintain better structural characteristics, and spatial and spectral fidelities when reconstructing the clean HSI. Qualitative and quantitative experiments are performed on publicly available datasets. The results demonstrate that SQAD outperforms the state-of-the-art methods of visual effect and objective evaluation metrics.

Band selection is an effective means to alleviate the curse of dimensionality in hyperspectral data. Many methods select a compact and low redundant band subset, which is inadequate as it may degrade the classification performance. Instead, more emphasis shall be put on selecting representative bands. In this article, we propose a robust unsupervised band selection method to address this issue. Our method reveals bandwise representativeness based on the comprehensive interband neighborhood structure. It incorporates an interband neighborhood graph into a sparse self-contained regression model in order to provide a reasonable measure for bandwise representativeness. The derived coefficient matrix not only uncovers bandwise importance values but also is coherent to the generalized interband local neighborhood structure. For constructing the interband neighboring structural graph, an integrated multigraph model is employed to achieve better generalization performance. It combines the benefit of multiple graphs but is insusceptible to the defects of a single one. To enhance the reliability of this model, a joint trace minimum and nonnegative constraint is imposed on the coefficient matrix. Accordingly, a multigraph integrated embedding and robust self-contained regression model (MGRSR) is formulated. In addition, an iterative update algorithm is developed to solve the problem. Comparative experiments on three hyperspectral data sets illustrate that MGRSR is robust to various data and has superior performance compared with several state-of-the-art methods.

Loop-closure detection is an indispensable module in the visual simultaneous localization and mapping (vSLAM) system. It typically consists of three main steps: image representation, loop-closure candidate selection, and loop-closure event verification. This article proposes a novel approach for loop-closure detection. In particular, we first introduce a lightweight convolutional auto-encoder network trained by the deep perceptual similarity loss for image representation. We then propose an image-to-sequence selection approach based on place sequence division and distance-weighted voting for loop-closure candidate selection. Furthermore, we propose a motion vector consensus constraint to improve locality preserving matching, which can be used for efficient loop-closure event verification that is robust for various complex environments. Extensive experiments have been conducted on four publicly available datasets. The results demonstrate that our method is able to achieve better recall performance than the state-of-the-art and meet the real-time requirement of vSLAM systems.

Loop-closure detection (LCD), which aims to recognize a previously visited location, is a crucial component of the simultaneous localization and mapping system. In this paper, a novel appearance-based LCD method is presented. In particular, we propose a simple yet surprisingly useful feature matching algorithm for real-time geometrical verification of candidate loop-closures, termed aslocal relative orientationmatching (LRO). It aims to efficiently establish reliable feature correspondences based on preserving local topological structures between the query image and candidate frame. To effectively retrieve candidate loop closures, we introduce the aggregated selective match kernel framework into the LCD task, which can effectively represent images and reduce the quantization noise of the traditional bag-of-words framework. In addition, the SuperPoint neural network is employed to extract reliable interest points and feature descriptors. Extensive experimental results demonstrate that our LRO can significantly improve the LCD performance, and the proposed overall LCD method can achieve much better performance over the current state-of-the-art on six publicly available datasets.

Autoencoder can not only extract features in an unsupervised manner, but also selects samples out that differs significantly from others. However, autoencoder is sensitive to noise and anomalies during training, and the relationships between pixels are discarded. In order to tackle these problems, we propose a robust graph autoencoder (RGAE) for hyperspectral anomaly detection. To be specific, we first redesign the objective function to encourage the network more robust to noise and anomalies. Meanwhile, a superpixel segmentation-based graph regularization term (SuperGraph) is incorporated into AE to preserve the geometric structure and spatial information simultaneously. Experiments with three real data sets are conducted to evaluate the performance, and the detection results demonstrate that our method outperforms other state-of-the-art hyperspectral anomaly detectors.

Recently, the learning-based method has received much attention in the unsupervised hyperspectral unmixing, yet their ability to extract physically meaningful endmembers remains limited and the performance has not been satisfactory. In this paper, we propose a novel two-stream Dirichlet-net, termed as uTDN, to address the above problems. The weight-sharing architecture makes it possible to transfer the intrinsic properties of the endmembers during the process of unmixing, which can help to correct the network converging towards a more accurate and interpretable unmixing solution. Besides, the stick-breaking process is adopted to encourage the latent representation to follow a Dirichlet distribution, where the physical property of the estimated abundance can be naturally incorporated. Extensive experiments on both synthetic and real hyperspectral data demonstrate that the proposed uTDN can outperform the other state-of-the-art approaches.

Since CapsNet [1] shattered all previous records of algorithms for image recognition, the capsule's conception has attracted bright attention. It interprets an object by the geometrical arrangement of parts. We think it can be transferred to hyperspectral images. In a hyperspectral data cube, each pixel spectrum can be regarded as a continuous curve representing its inherent properties. In the spatial domain, there are various spatial distributions in different positionsand there is usually a specific structural relationship between adjacently distributed categories. Based on HSI data's aforementioned structural characteristics, combined with the stacked capsule autoencoder, we propose our model to achieve an unsupervised HSI classification. In our model, the ConvLSTM is employed to discover part capsules of HSI, and we utilize Set Transformer to encode relations among all parts and indicate object capsules. The decoders of both phases use Gaussian mixture models to reconstruct specific information. Experimental results of the Pavia Center dataset show the exceptional of our model.

Loop closure detection (LCD), which aims to deal with the drift emerging when robots travel around the route, plays a key role in a simultaneous localization and mapping system. Unlike most current methods which focus on seeking an appropriate representation of images, we propose a novel two-stage pipeline dominated by the estimation of spatial geometric relationship. When a query image occurs, we select semantically similar images based on the SuperPoint network and the aggregated selective match kernel in the first stage, and then conduct robust geometric confirmation to verify true loop-closing pairs in the second stage. Based on the potential property of motion field in the LCD scene, a robust feature matching algorithm, termed as motion field consensus with locality preservation (MFC-LP), is proposed. In particular, we exploit the smoothness prior to guide the learning of the motion field for an image pair in a reproducing kernel Hilbert space (RKHS). Meanwhile, to enhance the local relevance of motion vectors, we design a locality preservation mechanism thus making the learned motion field more accurate. Extensive experiments on several publicly available datasets reveal that MFC-LP has a good performance in the general feature matching task and the proposed pipeline outperforms the current state-of-the-art approaches in the LCD task.

Anomaly detection (AD) has become a hotspot in hyperspectral imagery (HSI) processing due to its advantage in detecting potential targets without prior knowledge, and a variety of algorithms are proposed for a better performance. However, they usually either fail to extract intrinsic features underlying HSIs, or suffer from the contamination of noise and anomalies. To address these problems, we propose a new anomaly detector by integrating fractional Fourier transform (FrFT) with low rank and sparse matrix decomposition (LRaSMD). First, distinctive features of HSI data are extracted via FrFT. Then, row-constrained LRaSMD (RC-LRaSMD), which is more practical and stable than the traditional LRaSMD, is employed to separate background from noise and anomalies. Finally, we implement an atom-selection strategy to construct the background covariance matrix for detection. The experimental results with several HSI data sets demonstrate satisfying detection performance compared with other state-of-the-art detectors.

Hyperspectral band selection is of great value to alleviate the curse of dimensionality. For many band selection methods, however, the neglect of bandwise usefulness tends to result in the loss of valuable bands, but the retention of useless ones; consequently, this causes deterioration of the classification performance. In this sense, bandwise significance should be emphasized. To address this issue, this article proposes a manifold-preserving and weakly redundant (MPWR) unsupervised band selection method. In the method, a manifold-preserving band-importance metric is put forward to measure the bandwise essentiality. This ensures the retention of bands involving abundant intrinsic structures conductive to classification. Specifically, aimed at obtaining the presented band-importance metric, an attainment algorithm is presented, which mainly relies on the embedding learning and linear regression, followed by the introduction of multi-normalization combination. In addition, concerning the massive redundancy caused by the highly correlated bands, MPWR further establishes a constrained band-weight optimization model. Then, both bandwise manifold-preserving capability and intraband correlation are fully integrated into the band selection process. To solve the problem, a corresponding algorithm within the framework of the alternating direction method of multipliers (ADMM) is also developed. Regarding evaluating the effectiveness of the proposed method, comparative experiments with the state-of-the-art methods are conducted on three public hyperspectral data sets. Experimental results demonstrate the superiority and robustness of MPWR.

In this paper, we proposed a new end-to-end model, termed as dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Our method establishes an adversarial game between a generator and two discriminators. The generator aims to generate a real-like fused image based on a specifically designed content loss to fool the two discriminators, while the two discriminators aim to distinguish the structure differences between the fused image and two source images, respectively, in addition to the content loss. Consequently, the fused image is forced to simultaneously keep the thermal radiation in the infrared image and the texture details in the visible image. Moreover, to fuse source images of different resolutions, e.g., a low-resolution infrared image and a high-resolution visible image, our DDcGAN constrains the downsampled fused image to have similar property with the infrared image. This can avoid causing thermal radiation information blurring or visible texture detail loss, which typically happens in traditional methods. In addition, we also apply our DDcGAN to fusing multi-modality medical images of different resolutions, e.g., a low-resolution positron emission tomography image and a high-resolution magnetic resonance image. The qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our DDcGAN over the state-of-the-art, in terms of both visual effect and quantitative metrics.

Gaussian mixture model (GMM) can estimate not only the abundances and distribution parameters but also distinct end-member set for each pixel. However, the traditional GMM unmixing model only has proper smoothness and sparsity prior constraints on the abundances and thus cannot excavate the local spatial information in hyperspectral image (HSI). Thus, we propose a new unmixing method with superpixel segmentation (SS) and low-rank representation (LRR) based on GMM called GMM-SS-LRR, which can consider the local spatial correlation of HSI. First, we adopt the principal component analysis (PCA) to obtain the first principal component of HSI, which contains the most information for the entire HSI. Then, we adopt the SS in the first principal component of HSI to obtain the homogeneous regions, and the abundances in each homogeneous region have the underlying low-rank property. Finally, we unmix the pixels in each homogeneous region of HSI depending on the low-rank property of abundances. Experiments on synthetic datasets and real H-SIs demonstrate that the proposed GMM-SS-LRR is efficient compared with other current popular methods.

Image stitching stitches multiple overlapping images into a seamless image according to the corresponding geometric relationship between the reference and source images. In this study, the parallax-tolerant image stitching method based on robust elastic warping is applied to the stitching of drone images, and locality-preserving feature matching is used to effectively remove outliers from the drone images. The method can be divided into three stages, namely, locality-preserving feature matching, robust elastic warping, and global projectivity preservation. First, a set of high- precision point matching is provided for a drone image, and local matching is used. Second, the robust elastic warping function eliminates the parallax error, and the input image is distorted according to the calculated deformation on the grid plane. Finally, the global projectivity-preserving method is applied to obtain high-precision result panoramas. Experiments on several sets of drone images demonstrate that our method can generate better panoramas over the competitors.

Neural networks have been successfully used to extract deep features for many hyperspectral tasks. In this study, we propose a tiny effective model based on gate recurrent unit (GRU) with spectral-spatial information for hyperspectral image classification. In our method, the core GRU cell can learn interspectral correlations within an entirely continuous spectrum input, and spatial information is the initial state of this GRU cell as a prior. Experimental results demonstrate that our method can fully utilize spectral and spatial information to obtain competitive performance.

Deep neural networks have been successfully applied to extracting deep features for many hyperspectral tasks. Attention mechanism has been widely used in computer vision, inspired by this, we have designed a joint attention network for spectral-spatial classification of hyperspectral image. In our method, recurrent neural network (RNN) with attention can learn inner spectral correlations within a continuous spectrum, convolutional neural network (CNN) with attention is designed to focus on saliency features and spatial dependency in the neighbor regions. Experimental results demonstrate that our method can fully utilize spectral and spatial information to obtain competitive performance.

Recently, collaborative representation has received much attention in the hyperspectral image (HSI) classification due to its simplicity and effectiveness. However, the existing collaborative representation-based HSI classification methods ignore the correlation among different classes. To overcome this problem, we propose a discriminative kernel collaborative representation and Tikhonov regularization method (DKCRT) for HSI classification, which can make the kernel collaborative representation of different classes to be more discriminative. Specifically, the kernel trick is adopted to map the original HSI into a high space to improve the class separability. Besides, distance-weighted kernel Tikhonov regularization is adopted to enforce these training samples to have large representation coefficients, which are similar to the test sample in the high-dimensional feature space. Moreover, we add a discriminative regularization term to further enhance the separability of different classes, which can take the correlation among different classes into consideration. Furthermore, to take the spatial information of HSI into consideration, we extend the DKCRT to a joint version named JDKCRT. Experiments on real HSIs demonstrate the efficiency of the proposed DKCRT and JDKCRT.

Sparse unmixing (SU) of hyperspectral data have recently received particular attention for analyzing remote sensing images, which aims at finding the optimal subset of signatures to best model the mixed pixel in the scene. However, most SU methods are based on the commonly admitted linear mixing model, which ignores the possible nonlinear effects (i.e., nonlinearity), and the nonlinearity is merely treated as outlier. Besides, the traditional SU algorithms often adopt the$\ell _{2}$norm loss function, which makes them sensitive to noises and outliers. In this paper, we propose a robust SU (RSU) method with$\ell _{2,1}$norm loss function, which is robust for noises and outliers. Then, the RSU can be solved by the alternative direction method of multipliers. Finally, the experiments on both synthetic data sets and real hyperspectral images demonstrate that the proposed RSU is efficient for solving the hyperspectral SU problem compared with the state-of-the-art algorithms.

Infrared (IR) small target detection with high detection rate, low false alarm rate, and multiscale detection ability is a challenging task since raw IR images usually have low contrast and complex background. In recent years, robust human visual system (HVS) properties have been introduced into the IR small target detection field. However, existing algorithms based on HVS, such as difference of Gaussians (DoG) filters, are sensitive to not only real small targets but also background edges, which results in a high false alarm rate. In this letter, the difference of Gabor (DoGb) filters is proposed and improved (IDoGb), which is an extension of DoG but is sensitive to orientations and can better suppress the complex background edges, then achieves a lower false alarm rate. In addition, multiscale detection can be also achieved. Experimental results show that the IDoGb filter produces less false alarms at the same detection rate, while consuming only about 0.1 s for a single frame.

The generalized bilinear model (GBM) has been widely used for the nonlinear unmixing of hyperspectral images, and traditional GBM solvers include the Bayesian algorithm, the gradient descent algorithm, the semi-nonnegative-matrix-factorization algorithm, etc. However, they suffer from one of the following problems: high computational cost, sensitive to initialization, and the pixelwise algorithm hinders us from applying to large hyperspectral images. In this letter, we apply Nesterov's optimal gradient method to solve the least-square problem under the bound constraint, which is named as the bound projected optimal gradient method (BPOGM). The BPOGM can achieve the optimal convergence rate of$O(1/k^{2})$, with$k$denoting the number of iterations in BPOGM. We further apply the BPOGM to solve the GBM-based unmixing problem. Experiments on both synthetic data sets and real hyperspectral images demonstrate that the BPOGM is efficient for solving the GBM-based unmixing problem.

Recently, the sparse representation-based classification (SRC) methods have been successfully used for the classification of hyperspectral imagery, which relies on the underlying assumption that a hyperspectral pixel can be sparsely represented by a linear combination of a few training samples among the whole training dictionary. However, the SRC-based methods ignore the sparse representation residuals (i.e., outliers), which may make the SRC not robust for outliers in practice. To overcome this problem, we propose a robust SRC (RSRC) method which can handle outliers. Moreover, we extend the RSRC to the joint robust sparsity model named JRSRC, where pixels in a small neighborhood around the test pixel are simultaneously represented by linear combinations of a few training samples and outliers. The JRSRC can also deal with outliers in hyperspectral classification. Experiments on real hyperspectral images demonstrate that the proposed RSC and JRSRC have better performances than the orthogonal matching pursuit (OMP) and simultaneous OMP, respectively. Moreover, the JRSRC outperforms some other popular classifiers.