In the hyperspectral image (HSI) classification, convolutional neural networks (CNNs)-based approaches often struggle with the scarcity of labeled samples. The letter proposes an HSI classification method based on data expansion and consistency regularization with small samples. Specifically, we leverage the pixel-pair feature (PPF) to expand the dataset, which facilitates the adequate tuning of CNN parameters and alleviates the issue of overfitting. In addition, a designed CNN structure is employed to extract discriminative features from the limited number of labeled PPFs and numerous unlabeled PPFs. The CNN is trained via minimizing the weighted sum of supervised and unsupervised losses, where the supervised loss is calculated through the cross-entropy function while the unsupervised loss is evaluated with the consistency regularization item. Moreover, reliable references required in the consistency regularization item are provided after making an exponential moving average (EMA) on the outputs of CNNs at different training epochs. Ultimately, we conduct experiments on three real HSI datasets, and the results show that the proposed approach gains superior classification accuracy compared to several existing CNN-based approaches.

Infrared small-target detection (ISTD) in a single frame is an essential, yet challenging task due to its small size of targets, weak energy, and clutter background. Current methods either design complex network architectures to facilitate multilevel information interaction (e.g., DNA-Net and UIU-Net) or introduce structural texture priors to enhance feature discrimination (e.g., SRNet and CSRNet). However, both methods fail to explicitly distinguish or suppress the interference of complex background from infrared small targets, which makes them easy to “get lost” in clutter background with insufficient attention to the targets. In this work, we innovatively propose a novel background-masking approach (denoted as BGM) for ISTD. The proposed BGM aims to force the network to focus exclusively on the target by masking out irrelevant background information, thereby enhancing the network’s ability to detect weak and small infrared targets. Specifically, we present a new ISTD method that leverages a proxy training task with masking, enabling the network to simultaneously predict on both the original input and the masked data, where the background is randomly masked/forgotten. This strategy allows for a better concentration of the model on the shapeless targets rather than the cluttered background. The method is flexible with a simple U-shaped network without complicated manipulation and also computationally efficient without increasing the overall computational burden during inference. Extensive experiments demonstrate that our proposed BGM effectively enhances the detection performance of infrared small targets and achieves 70.8% mean intersection over union (mIoU) on IRSTD-1K. The source code would be available athttps://github.com/ZhihaoMa123/BGM

The classification performance of land features is strongly associated with the quality of samples. However, in the real world, the presence of class noise is inevitable. Class noise may seriously mislead the construction of model and limit the improvement of classifier performance. In response to this situation, a stratified mislabeled instances removal method based on the idea of density spatial clustering optimally (SD-SCM) is proposed. Herein, the convolutional neural network (CNN) is employed to evaluate the effectiveness of the proposed method.Besides, a famous noise filter, KNN-kernel Cluster based technology, is adopted to compare with SD-SCM. The results on two benchmark datasets, Indian Pines and Pavia University, demonstrate the effectiveness of the proposed noise removal method.

Remote sensing image fusion aims to generate high spatial resolution hyperspectral images (HR-HSI) by integrating low spatial resolution hyperspectral images (LR-HSI) and high spatial resolution multispectral images (HR-MSI). This paper propose a HSI-MSI fusion method based on the Pyramid Swin Transformer (PSTF). The pyramid design of the PSTF facilitates the extraction of multi-scale information, and it incorporates the Spatial-Spectral Crossed Attention (SSCA) module. The SSCA module is composed of the Cross Spatial Attention (CSA) module and the Spectral Feature Integration (SFI) module. The CSA module uses a cross-shaped self-attention mechanism, enhancing modeling flexibility for different spatial scales and non-local structures. The SFI module integrates global spectral information and local spatial-spectral correlation by introducing a global memory block, efficiently extracting and preserving spectral information. Compared with the state-of-the-art (SOTA) methods, experimental results demonstrate the effectiveness of the PSTF method.

Infrared and visible images with distinct and complementary information are fused to obtain more comprehensive information. However, traditional fusion algorithms often suffer from losing details and low fusion quality. Convolutional neural networks (CNN) have been proven to possess excellent feature extraction capabilities. Therefore, a fusion algorithm for infrared and visible image fusion based on CNN and Multi-Scale Transform (MST) is proposed in this paper. In this fusion algorithm, a dual-branch CNN is employed to map the original images to the weight map. The low-pass and high-pass bands obtained through multi-scale transformation are fused separately by combining the weight map with different fusion strategies. In the experiment, five MST algorithms and 21 pairs of images are tested. The experimental results demonstrate that the proposed fusion algorithm significantly enhances the fusion quality of the original MST algorithms.

High resolution range profile (HRRP) provides abundant target information but is susceptible to external electromagnetic interference. While infrared sensor possesses strong anti-jamming capability, it has limited detection range and is vulnerable to weather conditions, leading to reduced imaging resolution. The integration of radar and infrared sensors can synergize their respective strengths to not only improve the reliability and robustness of the system but also enhance the credibility and accuracy of the data. However, there exist many challenges in the research on the fusion of heterogeneous data like HRRP 1D data and infrared 2D data. In this letter, a radar infrared sensor fusion method based on hierarchical features mining (HFM) is proposed to solve the problems above. The method is applied to multi-target recognition tasks to verify the effectiveness. The results demonstrate that the proposed method can enhance the information completeness of the target and improve the accuracy of target recognition.

The classification of hyperspectral image (HSI) has become the focus of the remote sensing field. However, limited training data, which makes the classification task face a major challenge, is inevitable in remote sensing. To eliminate the negative effects of limited labeled samples, an enhanced ensemble method named RoXGBoost, which inherently combines Rotation Forest (RoF) and eXtreme Gradient Boosting (XGBoost) is proposed in this paper. This algorithm could increase the diversity of base classifiers by random feature selection and data transformation. Five ensemble learning methods, Random Forest (RF), AdaBoost, RoF, Rotation Boost and XGBoost, are applied as comparisons. The results on two benchmark datasets, Indian Pines and Pavia University, demonstrate the effectiveness of the RoXGBoost.

Multispectral and hyperspectral image fusion (MHF) aims to reconstruct high-resolution hyperspectral images by fusing spatial and spectral information. The block-item tensor fusion model is able to use endmember and abundance information to improve the quality of hyperspectral images. This paper implements image fusion based on a coupled non-negative block term tensor decomposition model. Firstly, the two abundance matrices are formed into a chunking matrix and L2,1-parametric is added as well, promoting structured sparsity and eliminating the scaling effect present in the model. Immediately after, the counter-scaling effect present in the model is eliminated by adding a L2-parametric number to the endmember matrix. Finally, the focus is on solving the noise/artifacts generated by the no exact estimation of rank in the model, and over-estimation of rank by coupling the chunking matrix and the endmember matrix together to reconstruct the matrix, adding L2,1-parameters to it to facilitate the elimination of chunks, and solving the problems using an extended iteratively reweighted least squares (IRLS) method. The experiments on the University of Pavia dataset show that the proposed algorithm works better compared to the state of the art methods.

Hyperspectral image super resolution (HSI-SR) problem aims to fuse a low-resolution hyperspectral image (HSI) with its corresponding multispectral image (MSI) to obtain a high-resolution hyperspectral image (HSR). However, the commonly used methods have some limitations. For example, the matrix decomposition method does not preserve the spatial or spectral information of the image well, and the tensor decomposition method has a high computational complexity. This paper proposes a method based on spectral subspace sparse tensor factorization (SSTF), which learns the spectral subspace from hyperspectral images, constrains this model using sparse tensor regularisation, transforms it to solve a convex optimisation problem, and iteratively optimises the problem using the alternating direction method of multipliers (ADMM). The computational complexity of the algorithm is effectively reduced while retaining spatial and spectral features. Compared with the state-of-the-art methods, experimental results demonstrate the effectiveness of the SSTF method.

An ensemble alignment subspace adaptation method is proposed in this letter for the cross-scene classification. It can settle the problem of both foreign objects in the same spectrum and different spectrums. The algorithm combines the idea of ensemble learning with the domain adaptive (DA) algorithm. Considering the sample imbalance problem of the original data (OD), the source data (SD) is obtained by multiple random sampling of OD according to certain rules and used as input. Then, geometric alignment and statistical alignment of SD and target data (TD) are performed to build a communal subspace, followed by the classification of TD. The classification labels are finally ensembled by counting the multiple classification results with retaining valid information. This technique can reduce the uncertainty and randomness of generating subspace projections. The experimental results on two real datasets show that the proposed algorithm has a terrific accuracy improvement compared with the traditional machine learning and DA methods.

The accuracy of a classifier, whether it is an ensemble or not, is directly influenced by the training data used in learning. In remote sensing, training data mislabeling is inevitable and faces a major challenge. This paper proposes a versatile data cleaning which handles the mislabeling problem by exploiting the ensemble concepts for identifying, then eliminating or correcting the mislabeled training data. A powerful ensemble method, random forest, is at the core of our filter design and helps to distinguish mislabeled data from uncorrupted data more accurately. The major contribution of this work lies on the explicit use of the hypothesis margin as a decision means to identify and eliminate or correct mislabeled training data in an ensemble learning framework. Another key development that makes our algorithm superior to existing approaches is a design that avoids rare class instances to be mistaken for class noise. This fundamental aspect makes our data cleaning system particularly suitable for remote sensing classification tasks which usually suffer from both mislabeling and imbalance problems. The effectiveness of our algorithm is demonstrated in performing mapping of land covers. The generalization performance of two major supervised noise-sensitive classifiers, boosting and K-nearest neighbors, is strengthened by effective class noise reduction. A comparative analysis is conducted with respect to random forest, deep convolutional neural networks, as well as two well-established ensemble-based class noise filters, the majority vote and the consensus vote filters. This analysis demonstrates that our approach is more accurate than deep convolutional neural networks (one-dimensional CNN, AlexNet, EfficientNet, ResNet50 and ShuffletNet) and the reference ensemble methods.

In this paper, the fusion method of the weighted median filter Gram-Schmidt transform transplants to the digital signal processor (DSP). Image fusion technology has always been a key technology in the field of remote sensing image processing, but the algorithm is rarely implemented on mobile devices, so the scope of use has great limitations. The algorithm in the paper blends multispectral images and panchromatic images in the same location. The multispectral image is filtered by using a weighted median filter, and then the processed image and the panchromatic image are fused through the Gram-Schmidt transform. The filtering process reduces noise interference in the image, and the fused image combines the advantages of both images with high resolution and high color information. Due to the portability of DSP chips, the algorithm can be mounted on many mobile devices. Reduce the process of data transfer and make the image processing process more convenient.

Pine wilt disease is one of the most dangerous forest diseases. Because of its strong infectivity and harm, it is very important to find out and stop it in time. In this paper, a novel spatial-spectral random forest (SRF) algorithm for pine wilt monitoring is proposed, for solving the problem of small manual detection range, long investigation time, and untimely discovery of the diseased tree. The proposed method organically combines spatial features with spectral information to quickly and efficiently mark the location of diseased trees. In this way, the online monitoring of the target area using the data of the Beijing-2 satellite is realized. This paper analyses the location of diseased trees and provides early warnings for disease-prone trees. The accuracy of the proposed algorithm is 86.66%, by the confusion matrix analysis.

International audience

With the continuous progress of computer deep learning technology, convolutional neural network (CNN), as a representative approach, provides a unique solution for hyperspectral image (HSI) classification. However, the parameters of CNN can not be well-tuned when the number of training samples is insufficient, resulting in unsatisfactory classification performance. To tackle the thorny problem, a deep ensemble CNN method based on sample expansion for HSI classification is studied in this paper. Specially, spatial information is first extracted and fused with original spectral bands to help classifiers obtain discriminant spectral-spatial features. Then we use the pixel-pair feature (PPF) to expand the number of training samples so that the parameters of CNN structure can be fully trained. In addition, deep ensemble CNN is employed in this paper, enabling the trained model to obtain better generalization ability and more robust classification results. Ultimately, the proposed method is applied to classify four widely used hyperspectral data sets. Experimental results show that the studied approach yields higher classification accuracy than some CNN-based methods even under the condition of small-size training set.

With the development of new active deception jamming, radar antijamming has become a major research hotspot, and the recognition of jamming type is one of its key steps. In recent years, deep learning has been successfully applied in the field of radar jamming recognition, such as convolutional neural networks (CNNs). However, it is difficult to effectively improve the accuracy of deep learning algorithms in the case of small sample. Furthermore, ensemble learning and transfer learning can effectively improve the model generalization performance. For the small sample problem, this article proposes a weighted ensemble CNN with transfer learning (WECNN-TL)-based radar active deception jamming recognition algorithm. The main idea of this method is to obtain the time–frequency distribution maps of jamming signals by the short-time Fourier transform (STFT), and then, their real parts, imaginary parts, moduli, and phases are combined differently to construct multiple datasets. Finally, an ensemble CNN (ECNN) model with weighted voting and transfer learning is constructed to realize jamming recognition. Experiments on the simulated and measured mixed datasets (including 12 types of samples) show that the proposed method can get better recognition performance than random forest (RF), support vector machine (SVM), and some CNN-based methods.

When airborne radar platforms have a hypersonic speed, the Doppler bandwidth will be several hundred times of that from the low speed platforms. There are contradictions between pulse repeat frequency (PRF), Doppler ambiguity and range swath during the system parameters design. Azimuth multichannel technique is applied to make the PRF lower and can get a wide range swath. The equivalent phase center (EPC) under high squint (HS) mode is calculated. Then the Doppler spectrum is reconstructed by spatial filtering method with azimuth dependent channel compensation.

In this paper, a novel forest disaster detection method based on fuzzy c-means (FCM) algorithm and genetic algorithm (GA) with neighborhood information (F-NGA) is proposed. The proposed method adopts FCM to pre-classify the original data first. Then, the neighborhood factors are added into the GA model to reclassify the results of FCM. Experiment results on two multispectral Formosat-2 forest images present that our algorithm obtains better detection performance when compared with FCM, fuzzy local information c-means (FLICM) and original GA algorithm.

Recently, convolutional neural network (CNN) is widely used in hyperspectral image classification (HSIC) because of its strong self-learning and efficient feature expression ability. However, the CNN model faces the “overfitting” problem when the number of training samples is small. To improve the classification accuracy of CNN under the condition of limited training set, an ensemble CNN method based on pixel-pair and random feature selection (RFS) for HSIC is proposed in this paper. With the purpose of expanding training samples, the pixel-pair feature (PPF) is used in the presented study. Besides, ensemble CNN based on RFS is applied to further improve the classification performance. Experimental results based on two standard hyperspectral images demonstrate that the proposed method achieves better classification performance than the PPF based on CNN (PPF-CNN) and RFS based on SVM (RFS-SVM) methods.

Convolution neural network (CNN) has been successfully applied to hyperspectral image classification. However, multiclass imbalance is a major problem in the classification of hyper spectral images, and traditional CNN can hardly improve the accuracy of minority classes effectively. In this paper, a new ensemble CNN with enhanced feature subspaces (ECNN-EFSs) algorithm is proposed, which utilizes an imbalanced training set to train the model and achieves accurate classification. Experimental results on two common hyperspectral datasets show that the proposed algorithm outperforms the traditional CNN and ensemble CNN algorithms.

The fusion of hyperspectral image (HSI) and light detection and ranging (LiDAR) data can provide complementary information and improve the accuracy of land cover classification. In this paper, a novel fusion method is proposed to fuse the HSI and LiDAR dataset based on multi-scale feature extraction and total variation. In the method, the extended multi-attribute profile (EMAP) is utilized to automatically extract structural information from HSI and LiDAR elements. The extracted features are then estimated in a lower-dimensional space by multi-scale total variation (MSTV). Finally, the classification map is generated by applying random forest classifiers on the fused data. In the experiment, the performance of the proposed method is evaluated on an urban dataset of Houston. The results demonstrate that classification accuracy could be significantly improved by the proposed method compared with other methods.

In this paper, a novel Synthetic Minority Oversampling Technique based Deep Rotation Forest(SMOTE-DRoF) algorithm is proposed for the classification of imbalanced hyperspectral image data. It builds a multi -level forests cascade model by training a balanced dataset generated by SMOTE. In this model, each level of the random forest produces misclassification information of the data which are used as guidance information to adjust the sample weight adaptively for the next level. Experiment results on the hyperspectral image Indian Pines AVRIS and University of Pavia ROSIS demonstrate that the proposed method can get better performance than support vector machine, random forest, rotation forest, SMOTE combined random forest, and SMOTE combined rotation forest in imbalance learning.

The characteristics of training data is a fundamental consideration when constructing any supervised classifier. Class mislabelling and imbalance are major training data issues that often adversely affect machine learning algorithms, including ensembles. This work proposes extended bagging algorithms to better handle noisy and multi-class imbalanced classification tasks. These algorithms upgrade the sampling procedure by taking benefit of the confidence in ensemble classification outcome. The underlying idea is that a bagging ensemble learning algorithm can achieve greater performance if it is allowed to choose the data from which it learns. The effectiveness of the proposed methods is demonstrated in performing classification on 10 various data sets.

The classification is one of the most important tasks of the hyperspectral remote sensing. However, the task always suffers from the curse of dimensionality which makes most classifier models disabled. In this paper, a novel ensemble method named feature separation based rotation forest (FSRoF) is proposed to avoid the influence of high-dimensionality by training a series of independent classifiers with the datasets in a low-dimensionality rotation space and using the out-of-bag instances to select the base classifiers of high quality to construct the final ensemble model. The random forest (RF) and the traditional rotation forest (RoF) are adopted as the comparisons in our experiment.

The presence of noise is often unavoidable and has been a serious nuisance factor that needs to be taken into account in the hyperspectral image classification. Effective noise handling is one of the most difficult problems in data classification. Ensemble-based filtering has been demonstrated successful in dealing with the class noise problem. In this paper, a novel two-step ensemble-based data filtering method is proposed to improve the hyperspectral image classification accuracy in the presence of class noise. The proposed method is a combination of noise redundancy classifiers and sensitive algorithms. The experimental results on two public hyperspectral datasets demonstrate the effectiveness of the proposed approach.

Convolutional neural networks (CNN) can automatically learn features from the hyperspectral image data, which could avoid the difficulty of manually extracting features. However, the number of training set for the classification of hyperspectral images is always limited, making it difficult for CNN to obtain effective features and resulting in low classification accuracy. In this paper, a spectral-spatial feature (SSF) extraction based CNN method is proposed for an accurate classification with a small training set. Experimental results based on two standard hyperspectral images demonstrate the effectiveness of the proposed method.

The problem of class imbalance is often encountered in remote sensing data and has a negative effect on the classification performance of supervised classifiers even in ensemble models. The ensemble margin is a fundamental concept in ensemble learning with potential effectiveness in improving the classification of remote sensing data. This paper proposes a novel margin based extended random forest algorithm to address the class imbalance issues in the difficult context of remote sensing classification. This algorithm combines ensemble learning with data sampling. A comparative analysis is conducted with respect to standard random forest, undersampling and over-sampling combined ensembles.

In this paper, we propose a novel ensemble margin based semi-supervised random forest (EMRF) algorithm for the classification of the hyperspectral image with limited training data. The proposed method tries to improve the effectiveness of the ensemble model via adaptively labeling the unlabeled instances with high classification probability then adding them into the training set. The classification probability of a training instance is reflected by the unsupervised margin value of this instance. The higher ensemble margin of an instance, the higher probability the instance being classified correctly and added into to the training set in the next iteration.

The accuracy of a supervised classifier is directly influenced by the quality of the training data used. However, real-world data often suffers from mislabelling issues. To handle the mislabeling problem, we propose an ensemble margin-based mislabeled training data identification, elimination and correction approach based on data ordering. A powerful ensemble method, random forest, is at the core of our algorithms design. The effectiveness of our methods is demonstrated in performing mapping of land covers. A comparative analysis is conducted with respect to the majority vote filter, a popular ensemble-based mislabeled data filter.

Rotation forest (RoF) is a powerful ensemble classifier and has been demonstrated the outstanding performance in hyperspectral data classification. However, the classification task suffers from the class imbalanced problem which has been considered to be one of the most important challenges. The traditional construction method of RoF biases classifying the majority classes and ignores recognizing the minority classes samples. This letter proposes a novel adaptive ensemble method based on SMOTE and RoF with differentiated sampling rates (AdaSRoF) for the multiclass imbalance problem. The proposed method adaptively generates several balanced data sets with more diversity and less noise by using SMOTE and a dynamic data sampling ratio for base classifiers. The obtained results on two publicly available hyperspectral images show that the proposed method can get more diversity and better performance than support vector machine (SVM), random forest (RF), and RoF in multiclass imbalance learning.

In this paper, we propose a novel Synthetic Minority Oversampling Technique based Rotation forest (SMOTERoF) algorithm for the classification of imbalanced hyperspectral image data. The main idea of the proposed method is to iteratively balance the class distribution of training set by SMOTE for each rotation decision tree. Experiment results on the hyperspectral image Indian Pines AVRIS with different imbalance ratio (IR) show that our algorithm obtains better classification performance compared with Rotation Forest (RoF), random undersampling, random oversampling, SMOTE, as well as Under sampling based RoF (UnderRoF) which is an extended version of UnderBagging.

In this letter, we propose a new weight-based rotation forest (WRoF) induction algorithm for the classification of hyperspectral image. The main idea of the new method is to guide the growth of trees adaptively via exploring the potential of important instances. The importance of a training instance is reflected by a dynamic weight function. The higher the weight of an instance, the more the next tree will have to focus on the instance. Experimental results on two real hyperspectral data sets show that the WRoF algorithm results in significant classification improvement compared with random forests and rotation forest.

Mislabeled training data is a challenge to face in order to build a robust classifier whether it is an ensemble or not. This work handles the mislabeling problem by exploiting four different ensemble margins for identifying, then eliminating or correcting the mislabeled training data. Our approach is based on class noise ordering and relies on the margin values of misclassified data. The effectiveness of our ordering-based class noise removal and correction methods is demonstrated in performing image classification. A comparative analysis is conducted with respect to the majority vote filter, a reference ensemble-based class noise filter.

In remote sensing, where training data are typically ground-based, mislabeled training data is inevitable. This work handles the mislabeling problem by exploiting the ensemble margin for identifying, then eliminating or correcting the mislabeled training data. The effectiveness of our class noise removal and correction methods is demonstrated in performing mapping of land covers. A comparative analysis is conducted with respect to the majority vote filter, a reference ensemble-based class noise filter.