The accurate and stable detection of infrared (IR) small targets is essential for long-range monitoring. However, current approaches struggle to effectively bridge the inherent semantic gaps between hierarchical features and emphasize their critical characteristics during multilevel feature fusion. To address this challenge, a new asymmetric intensive interactive fusion network (A$\text {I}^{2}$Net) is proposed to narrow the semantic gap and enhance feature refinement capability in multilevel feature fusion. A$\text {I}^{2}$Net employs a high-resolution network (HRNet) as its backbone, maintaining small target integrity in the network’s deeper layers through a high-resolution feature stream that preserves the information of raw small target. Furthermore, an asymmetric intensive interactive fusion (A$\text {I}^{2}$F) module based on differentiated asymmetric attention weight allocation is proposed to facilitate the interactive fusion of multilevel features and eliminate the semantic gap between hierarchical levels. After that, we develop a multiscale context feature extraction (MCFE) module to enrich spatial detail feature representation across multiple scales. The experimental results on the National University of Defense Technology Single-Frame InfraRed Small Target (NUDT-SIRST) Automatic Target Recognition key laboratory ground/air dataset (ATR ground/air dataset) demonstrate that A$\text {I}^{2}$Net achieves remarkable performance compared to the state-of-the-art methods.

High spatial resolution (HSR) image scene classification is currently a research hotspot, aiming to assign semantic labels based on visual information. However, many scene classification methods currently focus on studying images with small-size coverage, rendering them unsuitable for large-size image, thus limiting their applicability. To solve this problem, a dual-branch large-size image classification network (LSICNet) is proposed, to organize the input data pipeline into a nine-grid form for achieving scene classification for large-size HSR image. At first, a large-size remote sensing image classification (LSRSIC) dataset is constructed with full annotations, providing essential data support for scene-level classification. Secondly, the central image is input into the vision transformer (ViT) branch to enable the extraction of long-range dependency information. The masked autoencoders (MAE) branch is employed to learn the reconstruction of the masked central image through an encoder-decoder architecture, while the spatial contextual relationships between the masked regions and the visible neighboring image regions are explored, thereby effectively capturing the contextual features of the adjacent images. Finally, features from both branches are concatenated to achieve complementary fusion. Experiments on the proposed LSRSIC dataset demonstrate that the LSICNet obtains quite competitive classification results compared with some state-of-the-art methods.

Deep learning (DL) has emerged as a competitive method in single-modality-dominated remote sensing (RS) data classification tasks, but its classification performance inevitably encounters a bottleneck due to the lack of representation diversity in complicated spatial structures with various land cover types. Therefore, the RS community has been actively researching multimodal feature learning techniques for the same scene. However, expert annotation of multisource data consumes a significant amount of time and cost. This article proposes an end-to-end method called semisupervised multimodal dual-path network (SMDN). This method simultaneously explores spatial-spectral features contained in hyperspectral images (HSI) and elevation information provided by light detection and ranging (LiDAR). SMDN exploits an unsupervised novel encoder-decoder structure as the backbone network to construct a multimodal DL architecture by jointly training with a data-specific branch. To obtain discriminative multimodal representations, SMDN is able to guide the collaborative training of two different unsupervised features mapped in the latent subspace with limited labeled training samples. Furthermore, after a simple modification of the fusion strategy in SMDN, it can be applied to unsupervised classification problems. Experimental results on benchmark RS datasets validate the effectiveness of the developed SMDN compared over many state-of-the-art methods.

Accurate identification and delineation of teeth in cone-beam computed tomography (CBCT) images are crucial in the advancement of digital dentistry technology. Teeth exhibit high interclass similarity and often have fuzzy boundaries. In addition, it is difficult to obtain teeth samples due to the time-consuming annotation process. However, existing methods typically fail to incorporate this domain-specific prior information under limited labeled samples, which limits the improvement of segmentation performance. Based on the intrinsic characteristics of the tooth CBCT images, a self-supervised manifold transfer learning network (SMTLNet) is proposed to improve segmentation accuracy. Initially, an object-oriented self-supervised pretraining approach is designed to fully explore valuable image representations from unannotated images, and this helps reduce dependence on labeled samples. Furthermore, a manifold optimization strategy is employed to regularize the segmentation model to separate interclass samples while compacting intraclass neighbors. Finally, to address the issue of blurred tooth boundaries, a multiscale boundary constraint module is developed to extract multiscale boundary-aware features, and more discriminative tooth descriptions can be acquired in this way. The proposed SMTLNet method is evaluated on clinical datasets containing diverse challenging cases (e.g., impacted wisdom teeth, crowded dentition), and it achieves state-of-the-art performance with dice similarity coefficients (DSCs) of 91.8%/89.08% and Jaccard similarities (JSs) of 86.71%/82.87% under full (100%) and limited (20%) training data regimes, respectively. The method maintains anatomical precision with Hausdorff distances (HDs) of 1.41 mm (high-resource) and 2.35 mm (low-resource), demonstrating strong clinical applicability in digital dentistry workflows.

Hyperspectral imaging has become a popular imaging technique in the medical field, and the development of algorithms for computer-aided diagnosis (CAD) is urgently required. Traditional deep learning techniques require a lot of annotated data, which is a burden on doctors. Self-supervised learning (SSL) is a solution for extracting feature representations from unlabeled data. However, traditional CNN-based SSL algorithms cannot explore relations between neighboring and long-range spectral bands, which limits classification performance. In this letter, the proposed solution is a novel SSL method using a transformer-based technique called masked spectral bands modeling with shifted windows (MSBMSW). This method predicts masked spectral bands as the pretext task and uses a self-attention mechanism with shifted windows to capture the divergence of neighboring spectral bands and enhance information exchange between long-range spectral bands. Experimental results demonstrate that MSBMSW achieves better classification results than many state-of-the-art methods and has potential clinical value for CAD of MHSIs.

Scene classification of very high resolution (VHR) images is an active research subject in remote sensing community, and it has provided data or decision supports for many practical applications. Although existing CNN-based methods have achieved good classification results, they have not fully exploited rich potential information contained in pre-trained models. In this paper, a novel framework termed hierarchical features fusion of convolutional neural network (HFFCNN) is developed for scene classification of VHR images. On the whole, the HFFCNN covers two parallel modules to severally process convolutional features and fully connected (FC) features. At the first module, an adaptive spatial-wise attention based multi-scale nonlinear bag-of-visual-words (ASA-MNBoVW) model is designed to encoding convolutional feature maps, and the responses of discriminative regions are highlighted without introducing any additional parameters. For the second module, a weighted image pyramid structure is adopted to reveal geometric information and spatial layouts by aggregating local image patch-based FC features. Finally, these hierarchical features are combined for mutually complementing, and a linear classifier is adopted to predict semantic labels. Experimental results organized on two challenging data sets prove that the developed HFFCNN approach obtains more dramatic performance of scene classification than some state-of-the-art methods in terms of OAs.

Scene classification is an important research topic in the field of remote sensing (RS), and deep features from convolutional neural networks (CNNs) have shown good classification performance. However, a key issue is how to effectively combine context features for further improving classification accuracy. In this letter, an end-to-end framework termed deep neural network combined with context features (CFDNN) is proposed for scene classification. At first, the pretrained VGG-16 is transferred as feature extractor to obtain convolutional features. Then, two parallel modules, global average pooling (GAP) and long short-term memory (LSTM), are employed to extract global features and context features, respectively. Finally, a weighted concatenation method is introduced to combine the global and context features. As a result, the CFDNN method can adapt high spatial resolution (HSR) images with arbitrary size and obtain satisfactory classification accuracy. The experimental results on the aerial image data set (AID) demonstrate that the proposed CFDNN method has competitive classification performance compared with some state-of-the-art methods.

Scene classification is an indispensable part of remote sensing image interpretation, and various convolutional neural network (CNN)-based methods have been explored to improve classification accuracy. Although they have shown good classification performance on high-resolution remote sensing (HRRS) images, discriminative ability of extracted features is still limited. In this letter, a high-performance joint framework combined CNNs and vision transformer (ViT) (CTNet) is proposed to further boost the discriminative ability of features for HRRS scene classification. The CTNet method contains two modules, including the stream of ViT (T-stream) and the stream of CNNs (C-stream). For the T-stream, flattened image patches are sent into pretrained ViT model to mine semantic features in HRRS images. To complement with T-stream, pretrained CNN is transferred to extract local structural features in the C-stream. Then, semantic features and structural features are concatenated to predict labels of unknown samples. Finally, a joint loss function is developed to optimize the joint model and increase the intraclass aggregation. The highest accuracies on the aerial image dataset (AID) and Northwestern Polytechnical University (NWPU)-RESISC45 datasets obtained by the CTNet method are 97.70% and 95.49%, respectively. The classification results reveal that the proposed method achieves high classification performance compared with other state-of-the-art (SOTA) methods.

Hyperspectral image (HSI) classification is a hot topic in the field of remote sensing, and convolutional neural networks (CNNs) have shown good classification performance because of their capabilities of feature extraction. However, traditional CNN-based methods require a lot of labeled data during their training process, although the acquisition of labeled samples is complicated and time-consuming. In addition, a key issue for HSI classification is how to effectively explore the correlation within the spectral dimension and emphasize important spectral bands. In this letter, an end-to-end framework named spectral attention-based self-supervised CNN (SASCNN) is put forward for HSI classification. At first, the SASCNN takes raw 3-D cubes as input data, and a spectral attention module (SAM) is used to adaptively optimize channel-wise characteristics by adjusting the importance among continuous spectral bands. Then, by flexibly adding multilayer concatenation to integrate shallow and abstract features, the designed encoder–decoder part can be used to learn discriminative features and reproduce the inputs in a self-supervised manner. Experiments over the Heihe and Houston datasets demonstrate the effectiveness of the proposed self-supervised learning method.

Deep learning (DL) has received extensive attention from the remote sensing community in recent years due to its ability to learn deep abstract information through a hierarchical network. However, most DL methods fail to explore the local geometric structure relationship between samples within hyperspectral imagery (HSI) to improve feature extraction performance. To address this issue, a novel DL approach, termed deep manifold reconstruction neural network (DMRNet), is proposed in this letter. By introducing a graph embedding framework, DMRNet calculates a reconstruction point of each sample with corresponding neighbors and then constructs a graph model to discover the intrinsic manifold structure in HSI. On this basis, DMRNet develops a joint loss function to reduce the difference between actual and predictive values, and to explore the separability of the extracted deep features. Experimental results on real-world HSI data sets exhibit the superiority of DMRNet to some state-of-the-art methods.

Recently, the convolutional neural network (CNN) has made great progress in hyperspectral image (HSI) classification because of its powerful feature extraction capability. However, the standard CNN based on grid sampling neglects the inherent relation between HSI data, which leads to poor regional edge delineation and generalization ability. Graph convolutional network (GCN) has been successfully applied to data representation in a non-Euclidean space, and it can extract discriminative embedded features by dynamically updating irregular graphs. In this letter, we propose a novel method termed attention mechanism-based dual-path convolutional network (AMDPCN), which is composed of a GCN-based global information learning model (GILM) and a CNN-based local feature extraction network (LFEN). Specifically, AMDPCN fuses the global spatial relationships explored by GILM and the local discriminant features extracted by LFEN with three different strategies: addition, multiplication, and concatenation. Furthermore, a multi-scale attention mechanism (MS-AM) is developed to mitigate the Hughes phenomenon by adaptive recalibrating the nonlinear interdependence among the features. Experiments on Kennedy Space Center and Indian Pines data sets demonstrate the advantages of the proposed AMDPCN to state-of-the-art methods.

Deep learning-based methods have demonstrated their competitive classification performance with sufficient labeled training samples. However, in practical hyperspectral image (HSI) classification applications, the labeled samples available for training are extremely limited compared with a large amount of unlabeled data, because the expert annotation of HSI is labor-intensive and time-consuming. To address the abovementioned issues, an end-to-end framework called semisupervised spatial–spectral dual-path networks (S3DPN) is proposed to learn discriminative spatial–spectral features from limited labeled data and abundant unlabeled data. Unlike many semisupervised deep learning methods that require to produce pseudo-labels (cluster labels), an unsupervised branch of S3DPN can directly extract deep representations from unlabeled samples, and it utilizes octave convolution (Oct-Conv) to simultaneously mine local detail features and global contextual information of unlabeled samples. S3DPN improves classification results by exploring the fusion features to reconstruct supervised and unsupervised features in turn. Furthermore, a spatial–spectral attention mechanism is employed to take full advantage of supervised features to selectively emphasize effective unsupervised representations and suppress useless ones. Experimental results on three real HSI datasets demonstrate the superior classification performance of the proposed S3DPN compared with many state-of-the-art (SOTA) methods.

Building extraction is a critical part of remote sensing (RS) image interpretation, and it is a popular research topic in the RS community. However, building extraction from RS images is a difficult task due to its various shape, size, and complex scene. The extracted feature of existing deep learning methods is lack of discrimination, resulting in incomplete buildings and irregular boundaries. Most studies are mainly concentrated on urban areas, ignoring illegal blue-roofed building extraction in rural areas. To address the above-mentioned problems, a channel-spatial attention-based encoder-decoder network (CSA-UNet) is proposed for rural blue-roofed building extraction tasks from RS images. To extract the key areas of buildings, the CSA-UNet employed channel-spatial attention to the fused features of encoder and decoder for achieving discriminative and attentive features. At the same time, considering the problem of false negative predictions, a joint loss function is designed by giving weight to positive samples to alleviate this problem and optimize the CSA-UNet model. Furthermore, blue-roofed buildings are a special type of illegal building, so we take blue-roofed buildings as an example to carry out related research. And a blue roof dataset termed UAVBlue is built through unmanned aerial vehicle (UAV). Experimental results exhibit that the CSA-UNet is better than some state-of-the-art (SOTA) methods.

Deep color trackers mainly use pre-trained convolutional neural networks for classification and regression, but it is difficult to discriminate targets in complex backgrounds for its limited spectral information. Compared with color video, hyperspectral videos provide better discriminative ability due to the abundant material-based information. However, it is hard to train a robust deep model for hyperspectral videos. The key issues are that there exists much redundant information in hyperspectral videos and the training samples are inadequate. In this paper, a new background aware hyperspectral tracking (BAHT) method is designed for hyperspectral tracking task. Our method firstly designs a background aware band selection module to preserve bands that can better recognize a target from backgrounds. Then the selected bands are input to backbone networks, which are pre-trained on color videos, to describe the appearances of targets with deep semantic features. Experiments on hyperspectral video tracking dataset illustrate the good performance of BAHT tracker compared with popular color and hyperspectral trackers.

Scene classification of high-resolution images is an active research topic in the remote sensing community. Although convolutional neural network (CNN)-based methods have obtained good performance, large-scale changes of ground objects in complex scenes restrict the further improvement of classification accuracy. In this letter, a global–local dual-branch structure (GLDBS) is designed to explore discriminative features of the original images and the crucial areas, and the strategy of decision-level fusion is applied for performance improvement. To discover the crucial area of the original image, the energy map generated by CNNs is transformed to the binary image, and the coordinates of the maximally connected region can be obtained. Among them, two shallow CNNs, ResNet18 and ResNet34, are selected as the backbone to construct a dual-branch network, and a joint loss is designed to optimize the whole model. In the GLDBS, the two streams employ the same structure (ResNet18-ResNet34) as the backbone, while the parameters are not shared. Experimental results on the aerial image data set (AID) and NWPU-RESISC45 datasets prove that the proposed GLDBS method achieves remarkable classification performance compared with some state-of-the-art (SOTA) methods. The highest overall accuracies (OAs) on the AID and NWPU-RESISC45 datasets are 97.01% and 94.46%, respectively.

Hyperspectral images (HSIs) not only possess abundant spectral features but also present a detailed spatial distribution of land cover, and they have significant advantages in the fine classification of ground materials. Recently, using convolutional neural networks (CNNs) to extract spectral–spatial features has become an effective way for HSI classification. However, conventional convolution kernels learn features from fixed regular square regions, and rich spatial information has not been effectively explored. In this letter, an end-to-end model named spectral–spatial residual graph attention network (S2RGANet) is developed for HSI classification, and it has two crucial elements, including spectral residual and graph attention convolution modules. At first, two spectral residual modules are employed to capture discriminant spectral features. Then, graphs are constructed to reveal the relationship between points in local neighborhoods. By graph attention mechanism, local spatial information is adaptively aggregated from neighboring nodes. Experiments on two public HSI datasets demonstrate that the S2RGANet is significantly superior to some state-of-the-art (SOTA) methods with limited training samples.

Recently, the sparse representation (SR)-based graph embedding method has been extensively used in feature extraction (FE) tasks, but it is hard to reveal the complex manifold structure and multivariate relationship of samples in the hyperspectral image (HSI). Meanwhile, the small size sample problem in HSI data also limits the performance of the traditional SR approach. To tackle this problem, this article develops a new semisupervised FE algorithm called a geodesic-based sparse manifold hypergraph (GSMH). The presented method first utilizes the geodesic distance to measure the nonlinear similarity between samples lying on manifold space and further constructs the manifold neighborhood of each sample. Then, a geodesic-based neighborhood SR (GNSR) model is designed to explore the multivariate sparse correlations of different manifold neighborhoods. Considering the multivariate sparse manifold correlations among samples, a pair of semisupervised hypergraphs (HGs) is constructed to effectively incorporate the labeled and unlabeled training information in the embedding process and obtain the nonlinear discriminative feature representation for HSI. Experimental results on three HSI datasets indicate that the proposed method not only achieves satisfying FE performance with limited labeled training samples but also shows superiority compared with other state-of-the-art methods.

Feature extraction (FE), an important preprocessing step in hyperspectral image (HSI) classification, has received growing attention in the remote sensing community. In recent years, the FE ability of deep learning (DL) methods has been widely recognized. However, most DL models focus on training networks with strong nonlinear mapping ability. They fail to explore the intrinsic manifold structure in HSI, and their performance depends on large size of the labeled training set. To address the above problems, a novel FE approach, termed manifold learning-based semisupervised neural network (MSSNet), was proposed in this article. By introducing the graph embedding (GE) framework, MSSNet develops a semisupervised graph model to explore the manifold structure in HSI with both labeled and unlabeled data. On the basis of this graph model, MSSNet constructs a combined loss function to take into account the metric of difference values and the exploration of manifold margins; thus, it reduces the difference between the predictive value and the actual value to enhance the separability of the features extracted by the network. Experiments conducted on real-world HSI datasets demonstrate that the performance of the proposed MSSNet outperforms some related state-of-the-art FE approaches.

Scene classification is an active research topic in the remote sensing community, and complex spatial layouts with various types of objects bring huge challenges to classification. Convolutional neural network (CNN)-based methods attempt to explore the global features by gradually expanding the receptive field, while long-range contextual information is ignored. Vision transformer (ViT) can extract contextual features, but the learning ability of local information is limited, and it has a large computational complexity simultaneously. In this article, an end-to-end method is exploited by employing ViT as an excellent teacher for guiding small networks (ET-GSNet) in the remote sensing image scene classification. In the ET-GSNet, ResNet18 is selected as the student model, which integrates the superiorities of the two models via knowledge distillation (KD), and the computational complexity does not increase. In the KD process, the ViT and ResNet18 are optimized together without independent pretraining, and the learning rate of teacher model gradually decreases until zero, while the weight coefficient of the KD loss module is doubled. Based on the above procedures, dark knowledge from the teacher model can be transferred to the student model more smoothly. Experimental results on the four public remote sensing datasets demonstrate that the proposed ET-GSNet method possesses the superior classification performance compared to some state-of-the-art (SOTA) methods. In addition, we evaluate the ET-GSNet on a fine-grained ship recognition dataset, and the results show that our method has good generalization for different tasks in terms of some metrics.

The spatial heterogeneity is an important indicator of the malignancy of lung nodules in lung cancer diagnosis. Compared with 2D nodule CT images, the 3D volumes with entire nodule objects hold richer discriminative information. However, for deep learning methods driven by massive data, effectively capturing the 3D discriminative features of nodules in limited labeled samples is a challenging task. Different from previous models that proposed transfer learning models in a 2D pattern or learning from scratch 3D models, we develop a self-supervised transfer learning based on domain adaptation (SSTL-DA) 3D CNN framework for benign-malignant lung nodule classification. At first, a data pre-processing strategy termed adaptive slice selection (ASS) is developed to eliminate the redundant noise of the input samples with lung nodules. Then, the self-supervised learning network is constructed to learn robust image representations from CT images. Finally, a transfer learning method based on domain adaptation is designed to obtain discriminant features for classification. The proposed SSTL-DA method has been assessed on the LIDC-IDRI benchmark dataset, and it obtains an accuracy of 91.07% and an AUC of 95.84%. These results demonstrate that the SSTL-DA model achieves quite a competitive classification performance compared with some state-of-the-art approaches.

Scene classification of high spatial resolution (HSR) images can provide data support for many practical applications, such as land planning and utilization, and it has been a crucial research topic in the remote sensing (RS) community. Recently, deep learning methods driven by massive data show the impressive ability of feature learning in the field of HSR scene classification, especially convolutional neural networks (CNNs). Although traditional CNNs achieve good classification results, it is difficult for them to effectively capture potential context relationships. The graphs have powerful capacity to represent the relevance of data, and graph-based deep learning methods can spontaneously learn intrinsic attributes contained in RS images. Inspired by the abovementioned facts, we develop a deep feature aggregation framework driven by graph convolutional network (DFAGCN) for the HSR scene classification. First, the off-the-shelf CNN pretrained on ImageNet is employed to obtain multilayer features. Second, a graph convolutional network-based model is introduced to effectively reveal patch-to-patch correlations of convolutional feature maps, and more refined features can be harvested. Finally, a weighted concatenation method is adopted to integrate multiple features (i.e., multilayer convolutional features and fully connected features) by introducing three weighting coefficients, and then a linear classifier is employed to predict semantic classes of query images. Experimental results performed on the UCM, AID, RSSCN7, and NWPU-RESISC45 data sets demonstrate that the proposed DFAGCN framework obtains more competitive performance than some state-of-the-art methods of scene classification in terms of OAs.

In this paper, we develop a CNN-GCN joint network (CGJNet) to learn global scene features and context information of high resolution remote sensing (HRRS) images. The proposed CGJNet method is composed of two streams, including CNN-stream (C-stream) and GCN-stream (G-stream). In the C-stream, a variation of the DenseNet-121 is developed to describe global visual information of HRRS images. In the G-stream, a GCN model is designed to reveal spatial structure by constructing adjacency graphs. As a result, accuracy of scene classification is effectively improved via integrating the two parts of crucial information. Experimental results on the AID data set demonstrate that the proposed CGJNet framework achieves remarkable classification results compared with many state-of-the-art (SOTA) methods, and the highest OA reaches 97.14%.

In this letter, a new semisupervised dimensionality reduction (DR) method, termed geodesic-based manifold joint hypergraphs (GMJHs), is proposed for hyperspectral image (HSI). This method first builds a geodesic-based reconstruction model to discover the nonlinear similarity between two manifold reconstruction neighborhoods. Then, it implies the probabilistic relationship between unlabeled samples and each class via the geodesic-based reconstruction distance. With the probabilistic class relationship, a supervised hypergraph and an unsupervised hypergraph are constructed to represent the multivariate manifold relationship of samples. Finally, the supervised and unsupervised hypergraphs are jointed for learning optimal projection matrix and enhancing the intraclass compactness in low-dimensional embedding space. Experiments on two HSI data sets show that the proposed GMJH algorithm performs better performance than some state-of-the-art DR methods.

Hyperspectral image (HSI) generally contains a complex manifold structure and strong sparse correlation in its nonlinear high-dimensional data space. However, the existing manifold learning and sparse learning methods usually consider the manifold structure and sparse relationship separately rather than combining manifold and sparse properties to discover the intrinsic information in the original data. To simultaneously reveal the complex sparse relation and manifold structure of HSI, a novel feature extraction (FE) method, called local manifold-based sparse discriminant learning (LMSDL), has been proposed on the basis of manifold learning and sparse representation (SR). The LMSDL method first designs a new sparse optimization model called local manifold-based SR (LMSR) to reveal the local manifold-based sparse structure of data. Then, two geometrical sparse graphs are constructed to represent the discriminant relationship between samples and the geometrical and sparse neighbors. An objective function is constructed via geometrical sparse graphs and reconstruction points to learn a projection matrix for FE. The LMSDL effectively reveals the complex sparse relation and manifold structure in high-dimensional data, and it enhances the representation ability of extracted features for HSI classification significantly. The experimental results on the three real HSI datasets show that the proposed LMSDL algorithm possesses better performance in comparison with some state-of-the-art FE methods.

Sparse representation-based graph embedding methods have been successfully applied to dimensionality reduction (DR) in recent years. However, these approaches usually become problematic in the presence of the hyperspectral image (HSI) that contains complex nonlinear manifold structure. Inspired by recent progress in manifold learning and hypergraph framework, a novel DR method named local constraint-based sparse manifold hypergraph learning (LC-SMHL) algorithm is proposed to discover the manifold-based sparse structure and the multivariate discriminant sparse relationship of HSI, simultaneously. The proposed method first designs a new sparse representation (SR) model named local constrained sparse manifold coding (LCSMC) by fusing local constraint and manifold reconstruction. Then, two manifold-based sparse hypergraphs are constructed with sparse coefficients and label information. Based on these hypergraphs, LC-SMHL learns an optimal projection for mapping data into low-dimensional space in which embedding features not only discover the manifold structure and sparse relationship of original data but also possess strong discriminant power for HSI classification. Experimental results on three real HSI data sets demonstrate that the proposed LC-SMHL method achieves better performance in comparison with some state-of-the-art DR methods.

Deep neural networks have achieved great success in the field of image processing. The feature representation of RGB image can be easily obtained in spatial domain. Different from this, hyperspectral image (HSI) is a kind of high-dimensional data that contains rich spectral information. To explore the manifold structure in HSI, a new deep learning model termed deep manifold learning network (DMLN) was proposed in this paper. In DMLN, a graph based loss function is designed to combine the exploration of manifold structure and the extraction of deep abstract information, which can obtain the discriminant features by iteratively enhancing the compactness of intraclass samples and the separation of interclass samples. Experimental results on two real-world HSI data sets demonstrate the proposed DMLN outperformed some the state-of-the-art methods.

The great success of deep learning in hyperspectral imagery is attributed to the rapidly developing computational resources. Traditional deep learning methods generally use two different frameworks to learn spatial information and spectral information respectively, then stack deep features for classification. In this paper, a 3-D deep learning model named spatial-spectral combination convolutional neural network (SSCCNN) is proposed to extract discriminative spectral-spatial features. SS-CCNN is an end-to-end network, that is, the raw 3-D cubes can be used as input data without any preprocessing. SSCCN-N can learn the spatial-spectral features and combine shallow features and deep features to alleviate the declining -accuracy phenomenon. Experiments on University of Pavia and Indian Pines data set demonstrate SSCCNN can obtain higher classification accuracy than state-of-the-art methods.

Hyperspectral images (HSIs) possess a large number of spectral bands, which easily lead to the curse of dimensionality. To improve the classification performance, a huge challenge is how to reduce the number of spectral bands and preserve the valuable intrinsic information in the HSI. In this letter, we propose a novel unsupervised dimensionality reduction method called local neighborhood structure preserving embedding (LNSPE) for HSI classification. At first, LNSPE reconstructs each sample with its spectral neighbors and obtains the optimal weights for constructing the adjacency graph by modifying its loss function. Then, to discover the scatter information of the training samples, LNSPE minimizes the scatter between the pixels and the corresponding neighbors and maximizes the total scatter of the HSI data. Finally, it incorporates the scatter information and the dual graph structure to enhance the aggregation of the HSI. As a result, LNSPE can effectively reveal the intrinsic structure and improve the classification performance of the HSI data. The experimental results on two real hyperspectral data sets exhibit the efficiency and superiority of LNSPE to some state-of-the-art methods.

The scene classification of high spatial resolution (HSR) images is a challenging task in the remote sensing community. How to construct a discriminative representation of the HSR scene is a key step to improve classification performance. In this letter, we propose a novel feature extraction method termed multilayer feature fusion network (MF2Net) for scene classification. At first, the transferred VGGNet-16 model is employed as a feature extractor to acquire multilayer convolutional features. Then, several layers including pooling, transformation, and fusion layers are designed to process hierarchical features in four branches, and the prediction probability can be obtained for classification. Finally, the proposed model is optimized by fine-tuning techniques, where a novel data augmentation approach is explored to improve generalization ability. As a result, MF2Net effectively applies useful information from multilayers to improve the accuracy of scene classification. The experimental results on AID and NWPU-RESISC45 data sets exhibit that the MF2Net method obtains quite competitive classification results compared with many state-of-the-art methods.

The graph embedding (GE) methods have been widely applied for dimensionality reduction of hyperspectral imagery (HSI). However, a major challenge of GE is how to choose the proper neighbors for graph construction and explore the spatial information of HSI data. In this paper, we proposed an unsupervised dimensionality reduction algorithm called spatial-spectral manifold reconstruction preserving embedding (SSMRPE) for HSI classification. At first, a weighted mean filter (WMF) is employed to preprocess the image, which aims to reduce the influence of background noise. According to the spatial consistency property of HSI, SSMRPE utilizes a new spatial-spectral combined distance (SSCD) to fuse the spatial structure and spectral information for selecting effective spatial-spectral neighbors of HSI pixels. Then, it explores the spatial relationship between each point and its neighbors to adjust the reconstruction weights to improve the efficiency of manifold reconstruction. As a result, the proposed method can extract the discriminant features and subsequently improve the classification performance of HSI. The experimental results on the PaviaU and Salinas hyperspectral data sets indicate that SSMRPE can achieve better classification results in comparison with some state-of-the-art methods.

A key challenge in hyperspectral image (HSI) classification is how to effectively utilize the spectral and spatial information of limited labeled training samples in the data set. In this article, a new spatial-spectral combined classification method, termed local linear spatial-spectral probabilistic distribution (LSPD), has been proposed on the basis of local geometric structure and spatial consistency of HSI. LSPD extracts discriminating spatial-spectral information from limited labeled training samples and their spatial-spectral neighbors. Then, it constructs a multiclass probability map by exploiting the local linear representation and spatial information of HSI. Finally, the spatial-spectral weighted reconstruction has been performed on the probability map, and the class of test sample can be predicted by the maximum value of LSPD. LSPD not only exploits spectral information to discover more intrinsic properties of the labeled training data but also utilizes the spatial relationship between samples to effectively improve discriminating power for classification. Experimental results on the Indian Pines, PaviaU, and HoustonU hyperspectral data sets demonstrate that the proposed LSPD method possesses better classification performance by comparing with some state-of-the-art classifiers.

Marginal Fisher analysis (MFA) exploits the margin criterion to compact the intraclass data and separate the interclass data, and it is very useful to analyze the high-dimensional data. However, MFA just considers the structure relationship of neighbor points, and it cannot effectively represent the intrinsic structure of hyperspectral image (HSI) that possesses many homogenous areas. In this paper, we proposed a new dimensionality reduce (DR) model, termed local geometric structure Fisher analysis (LGSFA), for HSI classification. At first, this method computes the reconstruction point of each point with its intraclass neighbor points. Then, an intrinsic graph and a penalty graph are constructed to reveal the intraclass and interclass relationship, respectively. Finally, the neighbor points and corresponding reconstruction points are used to enhance the intraclass compactness and interclass separability in low-dimensional space. LGSFA can effectively reveal the intrinsic manifold structure and obtains the discriminating feature of HSI data. Experiments on Salinas HSI data set show that the proposed LGSFA algorithm performs the best classification results than other state-of-the-art methods.

To utilize the spatial information and manifold structure in hyperspectral image (HSI), we propose a spatial-spectral manifold reconstruction classifier (SSMRC) for HSI classification in this paper. The SSMRC method firstly uses a mean filter to combine the spatial neighborhood information. Then the manifold reconstruction error utilizes as a measurement of how well a data point resides on a manifold, and the class label can be assigned with the minimum reconstruction error. It makes full use of spatial information and discriminating manifold structure in HSI, and the classification ability is further improved. The effectiveness of the proposed method is verified on real HSI data set with promising results.

The graph embedding (GE) framework is very useful to extract the discriminative features of hyperspectral images (HSIs) for classification. However, a major challenge of GE is how to select a proper neighborhood size for graph construction. To overcome this drawback, a new semisupervised discriminative learning algorithm, which is called the semisupervised sparse manifold discriminative analysis (S3MDA) method, was proposed by using manifold-based sparse representation (MSR) and GE. The proposed algorithm utilizes MSR to obtain the sparse coefficients of labeled and unlabeled samples. Then, it constructs a within-class graph and a between-class graph using the sparse coefficients of labeled samples, as well as an unsupervised graph with the sparse coefficients of unlabeled samples. Finally, it uses these graphs to obtain a projection matrix for feature extraction (FE) of HSI in a low-dimensional space. The S3MDA method not only inherits the merits of MSR to reveal the sparse manifold properties of data but also enhances interclass separability and intraclass compactness to improve the discriminating power for classification. Extensive experiments on two real HSI data sets obtained with a reflective optics system imaging spectrometer and an airborne visible/infrared imaging spectrometer show that the proposed algorithm is significantly superior to other state-of-the-art FE methods in terms of classification accuracy.

Sparse manifold learning has drawn more and more attentions recently, and sparsity preserving projections (SPP) has been proposed, which inherits the advantages of sparse reconstruction. However, SPP only focuses on the sparse structure, ignoring the discriminant information of labeled samples. In this paper, we proposed a new supervised dimensionality reduction method, which is called sparse discriminant embedding (SDE), for hyperspectral image (HSI) classification. SDE utilizes the merits of both intermanifold structure and sparsity property. It not only preserves the sparse reconstructive relations through l1-graph but also enhances the intermanifold separability of data, and the discriminating power of SDE is further improved than SPP. Experiments on two real HSIs collected by the Airborne Visible/Infrared Imaging Spectrometer and Reflective Optics System Imaging Spectrometer sensors are performed to demonstrate the effectiveness of the proposed SDE method.

Locally linear embedding (LLE) depends on the Euclidean distance (ED) to select the k-nearest neighbors. However, the ED may not reflect the actual geometry structure of data, which may lead to the selection of ineffective neighbors. The aim of our work is to make full use of the local spectral angle (LSA) to find proper neighbors for dimensionality reduction (DR) and classification of hyperspectral remote sensing data. At first, we propose an improved LLE method, called local spectral angle LLE (LSA-LLE), for DR. It uses the ED of data to obtain large-scale neighbors, then utilizes the spectral angle to get the exact neighbors in the large-scale neighbors. Furthermore, a local spectral angle-based nearest neighbor classifier (LSANN) has been proposed for classification. Experiments on two hyperspectral image data sets demonstrate the effectiveness of the presented methods.

Feature selection has attracted a huge amount of interest in both research and application communities of hyperspectral image (HSI) classification. Generally, supervised feature selection methods are superior to unsupervised ones without label information. However, in classification of HSI, the labeled samples are often difficult, expensive or time-consuming to obtain. In this paper, we proposed a novel semi-supervised feature selection method, called Locality and Globality Discriminant Feature (LGDF), for HSI classification. This method combines Fisher's criteria and Graph Laplacian, which makes full use of both labeled and unlabeled data points to discover both manifold and discriminant structure in HSI data. In the proposed method, an optimal subset of features is identified if at this subset neighbor points or points sharing the same label are close to each other, while non-neighbor points or points with different labels are far away from each other. Experimental results on Washington DC Mall and AVIRIS Indian Pines hyperspectral datasets demonstrate the effectiveness of the proposed method.

A new manifold learning method, called parameter-free semi-supervised local Fisher discriminant analysis (pSELF), is proposed to map the gene expression data into a low-dimensional space for tumor classification. Motivated by the fact that semi-supervised and parameter-free are two desirable and promising characteristics for dimension reduction, a new difference-based optimization objective function with unlabeled samples has been designed. The proposed method preserves the global structure of unlabeled samples in addition to separating labeled samples in different classes from each other. The semi-supervised method has an analytic form of the globally optimal solution, which can be computed efficiently by eigen decomposition. Experimental results on synthetic data and SRBCT, DLBCL, and Brain Tumor gene expression data sets demonstrate the effectiveness of the proposed method.

An improved manifold learning method, called Uncorrelated Local Fisher Discriminant Analysis (ULFDA), for face recognition is proposed. Motivated by the fact that statistically uncorrelated features are desirable for dimension reduction, we propose a new difference-based optimization objective function to seek a feature submanifold such that the within-manifold scatter is minimized, and between-manifold scatter is maximized simultaneously in the embedding space. We impose an appropriate constraint to make the extracted features statistically uncorrelated. The uncorrelated discriminant method has an analytic global optimal solution, and it can be computed based on eigen decomposition. As a result, the proposed algorithm not only derives the optimal and lossless discriminative information, but also guarantees that all extracted features are statistically uncorrelated. Experiments on synthetic data and AT&T, extended YaleB and CMU PIE face databases are performed to test and evaluate the proposed algorithm. The results demonstrate the effectiveness of the proposed method.

Automatic face recognition is a challenging problem in the biometrics area, where the dimension of the sample space is typically larger than the number of samples in the training set and consequently the so-called small sample size problem exists. Recently, neuroscientists emphasized the manifold ways of perception, and showed the face images may reside on a nonlinear submanifold hidden in the image space. Many manifold learning methods, such as Isometric feature mapping, Locally Linear Embedding, and Locally Linear Coordination are proposed. These methods achieved the submanifold by collectively analyzing the overlapped local neighborhoods and all claimed to be superior to such subspace methods as Eigenfaces and Fisherfaces in terms of classification accuracy. However, in literature, no systematic comparative study for face recognition is performed among them. In this paper, we carry out a comparative study in face recognition among them, and the study considers theoretical aspects as well as simulations performed using CMU PIE and FERET face databases.