Continual learning (CL) is crucial for advancing human-level intelligence, but its theoretical understanding, especially regarding factors influencing forgetting, is still relatively limited. This work aims to build a unified theoretical framework for understanding CL using feature learning theory. Different from most existing studies that analyze forgetting under linear regression model or lazy training, we focus on a more practical two-layer convolutional neural network (CNN) with polynomial ReLU activation for sequential tasks within a signal-noise data model. Specifically, we theoretically reveal how the angle between task signal vectors influences forgetting that: acute or small obtuse angles lead to benign forgetting, whereas larger obtuse angles result in harmful forgetting. Furthermore, we demonstrate that the replay method alleviates forgetting by expanding the range of angles corresponding to benign forgetting. Our theoretical results suggest that mid-angle sampling, which selects examples with moderate angles to the prototype, can enhance the replay method’s ability to mitigate forgetting. Experiments on synthetic and real-world datasets confirm our theoretical results and highlight the effectiveness of our mid-angle sampling strategy.

Remote sensing (RS) modalities from multi-sensor platforms, including hyperspectral (HS) and light detection and ranging (LiDAR) data, have been garnering increasing attention in overcoming the lack of information diversity. However, the specific modality correlation is always weakened and neglected. By reducing the spectral uncertainty, hyperspectral intrinsic image decomposition (HIID) has been proven to be effective in enhancing the HS image quality. It is an ill-posed problem and is challenged to capture the inherent modality connection. This paper proposes a shadow mask driven multimodal intrinsic image decomposition (smMIID) for HS and LiDAR data fusion. Notably, smMIID creatively clarifies and activates the potential of the crossmodal information in RS modalities and builds on the strength of the LiDAR elevation information when constructing HIID constraints. Our motivation is to overcome the deficiency of information diversity and modality correlation in existing IID based frameworks for better data fusion performance. Classic IID methods decompose the HS image into one reflectance component (RC) and one shading component (SC). There are three constraints in smMIID: 1) the fundamental constraint on RC and SC, 2) the HS-LiDAR hybrid gradient based constraint on RC and 3) the LiDAR gradient based constraint on SC. A shadow mask is obtained and utilized to guarantee spectral consistency when combining neighboring samples. Experiments on HS and LiDAR datasets prove that smMIID outperforms other techniques in terms of visualization and classification. The analyses are provided theoretically and experimentally to demonstrate that smMIID is robust to shadow mask generation.

Tridimensional empirical mode decomposition (TEMD) has been successfully employed as a powerful tool for hyperspectral classification. It achieves good performance to analyze nonlinear/non-stationary data and capture abstract features of hyperspectral images (HSIs). However, the major challenge is how to adequately exploit and balance decomposed edge-wise (EW) and trend-wise (TW) features. To address this issue, we propose an accelerated adaptive feature balance technique (A2FBT) based on TEMD. Due to complex discriminant information, A2FBT initially decomposes the HSI into varying oscillations adequately, including several tridimensional intrinsic mode functions (TIMFs) and one residual (RES). They contain high frequency EW and low frequency TW features, respectively. To further accelerate the decomposition, a k-d tree based search strategy is proposed for filter size selection. To bridge the gaps between the RES and all TIMFs, a novel adaptive feature balance strategy is presented by assigning adaptive trade-offs. We aim to balance multiple features adequately according to the intrinsic discriminant information. A combined metric based strategy is provided for trade-off calculation, measuring the mutual similarities between oscillations comprehensively. A2FBT pays more attention to these similarities, reinforcing the respective contributions of EW and TW features. Experiments on four typical hyperspectral datasets illustrate the effectiveness of A2FBT compared with several state-of-the-art techniques.

Hyperspectral image processing is faced with difficulties considering its redundant features and complex information. Studies on hyperspectral feature extraction in the deep learning domain have become increasingly popular. The mainstream techniques fully consider the spatial information in local neighborhoods when extracting spectral features by constructing deep neural networks. Deep generative models simulate the intrinsic structure of samples by adequately training, showing their potential values for signal processing. In this article, a convolutional two-stream network (cs2GAN-FE) based on the improved Wasserstein generative adversarial network (WGAN) is proposed for unsupervised hyperspectral spatial–spectral feature extraction. The improved WGAN is composed of one generator and one discriminator; the former perceives real data distributions, and the latter determines the attribution of generated data. The designed two-stream strategy is not a simple extension of a one-stream strategy and considers both the static spectral–spatial information and the dynamic spectral reflectance variation in multiple bands. Intrinsic spatial–spectral features are extracted by the trained discriminator considering sample distributions and feature relationships. The loss function is also improved for the unique structure of cs2GAN-FE. Various state-of-the-art techniques are chosen for comparison. Experimental results show the feasibility and potential of this network. Besides, experiments with the random split and the disjointed split both show that the proposed method can outperform other comparison techniques.

Abstract Hyperspectral unmixing is an important technique which attempts to acquire pure spectra of distinct substances (endmembers) and estimate fractional abundances from highly mixed pixels. This paper proposed a novel deep network‐based framework for unmixing problem. It contains two parts: a three‐dimensional convolutional autoencoder for hyperspectral denoising (denoising 3D CAE) which aims to recover data from highly noised input imagery through an unsupervised manner, and a restrictive non‐negative sparse autoencoder which extracts endmembers and abundances from the scene simultaneously. The proposed denoising 3D CAE network integrates 3D operations in each layer, which allows manipulating volumetric representation of the image data directly and facilitates hierarchical exploration for latent information. Being trained with corrupted hyperspectral image data, the denoising 3D CAE network has strong capacity of capturing the principle and robust local features in spatial and spectral domains efficiently, and it shows superior performance for image recovery with high noise disturbance. Moreover, a part‐based non‐negative autoencoder is concatenated, and the ‐norm penalty is imposed for sparsity enhancement of the solution. Comparative experiments are conducted both on synthetic and real‐world hyperspectral data, which demonstrate the applicability and effectiveness of the proposed unmixing framework.

Hyperspectral image with high dimensionality always increases the computational consumption, which challenges image processing. Deep learning models have achieved extraordinary success in various image processing domains, which are effective to improve classification performance. There remain considerable challenges in fully extracting abundant spectral information, such as the combination of spatial and spectral information. In this article, a novel unsupervised hyperspectral feature extraction architecture based on spatial revising variational autoencoder (AE) (UHfeSRVAE) is proposed. The core concept of this method is extracting spatial features via designed networks from multiple aspects for the revision of the obtained spectral features. Multilayer encoder extracts spectral features, and then, latent space vectors are generated from the obtained means and standard deviations. Spatial features based on local sensing and sequential sensing are extracted using multilayer convolutional neural networks and long short-term memory networks, respectively, which can revise the obtained mean vectors. Besides, the proposed loss function guarantees the consistency of the probability distributions of various latent spatial features, which obtained from the same neighbor region. Several experiments are conducted on three publicly available hyperspectral data sets, and the experimental results show that UHfeSRVAE achieves better classification results compared with comparison methods. The combination of spatial feature extraction models and deep AE models is designed based on the unique characteristics of hyperspectral images, which contributes to the performance of this method.

Feature selection, which is called band selection for hyperspectral data, is widely used for hyperspectral images. A novel hyperspectral band selection method based on combined fast and adaptive tridimensional empirical mode decomposition (cFATEMD) is proposed in this study. The hyperspectral data is decomposed into a set of tridimensional intrinsic mode functions (TIMFs) and a residual (RES) by FATEMD, which can reduce high‐frequency noise and signal. A stop condition of the decomposition is proposed based on the k‐means clustering algorithm and the Dunn validity index, which can prevent excessive decomposition and make generated RES contain as much useful information as possible. In consideration of the useful information in decomposition results, these TIMFs and the RES are combined into a new data based on the spectral similarity between themselves and the original data. Four state‐of‐the‐art band selection methods, cooperating with the proposed cFATEMD, are used to select bands by the new combined data. Several experiments are conducted on three publicly available hyperspectral datasets and the results are compared with corresponding methods’ results using the original data. Experimental results demonstrate that the proposed method yields great classification appearance.

Hyperspectral technology is a huge leap of remote sensing technology, however, how to make full use of its rich information is a difficult problem. A novel hyperspectral band selection method based on 3D empirical mode decomposition (3D-EMD) is proposed by constructing the upper approximation and the lower approximation of the whole image. The hyperspectral image (HSI) is decomposed into a set of tridimensional intrinsic mode functions (TIMFs) and bands are selected from the HSI using each TIMF based on optimum index factor (OIF). These bands are combined into a band combination. Experimental results demonstrate that the proposed method yields improved decomposition performance on the HSI and increases classification accuracy.

Hyperspectral data with high dimensionality always needs more storage space and increases the computational consumption, manifold learning is extensively used in dimensionality reduction. A novel dimensionality reduction method based on manifold learning is proposed through learning a stable local manifold representation. Four adjacency graphs are constructed to model the interclass similarity, interclass diversity, intraclass similarity and intraclass diversity respectively, and then merge these graphs into the discriminant objective function for linear dimensionality reduction. The classification results in the use of different methods are compared with and experimental results show that the proposed method is effective and it is superior to comparison methods.

In order to overcome the noise impact of hyperspectral images during the acquisition process due to target movement and environmental interference, this paper proposes a spectrum elaboration method based on empirical mode decomposition (EMD) for visible and near-infrared hyperspectral image data. The difference with the prior arts is that this method obtains a spectral curve by processing a series of hyperspectral data cubes. The experiments show that this method can obtain the vector of spectral signature that reflects the essential features for the moving object, as the results of spectral angle indicate that discrepancies between non-congeners are greater and similar congeners are more similar, which compared to the gaussian filtering method and the direct averaging method. has a higher accuracy than the traditional method without feature enhancement and has good innovative and practical value.

A novel deep learning classification method for hyperspectral data based on convolutional neural network is proposed in this paper. Deep learning means bringing multiple layers instead of one to the structure. Through convolution layers and pooling layers, the features in different layers are extracted from original spectral feature images. The key of this method is to restructure spectral feature images and choose convolution filters with a reasonable size, so that the spectral features of different land coverings in high dimensions can be extracted properly. In our experiments, proposed method was applied for hyperspectral data in several different situations, and preferable classification performance were obtained through relative parameters adjustment, which were given recommended scope during our comparative experiments.

Manifold learning based unsupervised classification methods will be unable to obtain satisfactory results because of the lack of training samples. The employment of training samples' information makes manifold learning based classification become supervised, and thus brings the improvement on classification accuracy. In order to make full use of this information, we emphatically consider the hyperspectral data distribute by clusters. A novel supervised manifold learning method termed class encoding is proposed for hyperspectral data classification. The experimental results show that this algorithm has better classification performance than the existing supervised manifold learning algorithm.

To address the multi-classification problems of hyperspectral dataset, a new method with weighted kernel function based on Chernoff distance is proposed. Chernoff distance utilizes the information between categories and strengthens the separability of original dataset. The adjustable parameter in Chernoff distance can fit the hyperspectral dataset well compared with other least upper bounds. Pairwise decision tree reduces the number of subclassifiers that the dataset requires and improves the classification accuracy. The guidance of the weighed subclassifiers is global separability metric computed by Chernoff distance. Weighted subclassifiers highlight bands with more useful information and reduce accumulative error. Comparative experiment shows the effectiveness of the proposed method.