Several spectral and spatial dictionary-based methods exist for fusing a high-spatial-resolution multispectral image (HR-MSI) with a low-spatial-resolution hyperspectral image (LR-HSI). However, using only one type of dictionary is insufficient to preserve spatial and spectral information simultaneously, while utilizing both dictionaries would increase the computational costs. To address this problem, we propose a fast fusion method (called FFD) for HR-MSIs and LR-HSIs via subpixel-shift decomposition. In this method, through joint optimization of low rank and sparsity within the framework of subpixel shift and sparse representation, an ultimate spectral dictionary is acquired along with its associated coefficients. Specifically, the HR-MSI is decomposed into subimage sequences of the same spatial resolution as the LR-HSI first, to replace the use of spatial dictionaries. Subsequently, a new fusion model is constructed based on this decomposition incorporating the constraints of low rank and sparsity, and especially, a low-rank term is introduced to constrain the spectral consistency along the decomposition direction. Then, the model is theoretically derived by using the alternating direction method of multipliers (ADMM) method, and a joint optimization for the spectral dictionary and its sparse coefficients is obtained. Finally, the desired HR-HSI can be reconstructed by using the above fused subimages through a simple inversed composition. Experimental results on different datasets show that compared with the other related methods, our proposed FFD can achieve an equivalent fusion effect to the best of them in a much shorter time.

Public spectral library that can provide authentic and reliable spectral information has been widely used in various applications, especially in the spectral library based spectral super-resolution. In general, the joint of multisource spectral libraries from different organizations can provide more varied and richer spectral information, but the differences in collection conditions and equipment between multisource spectra may greatly affect their application effectiveness. To cope with this problem, an integration method of multisource spectral libraries via benchmark alignment for spectral super-resolution is proposed. In this method, a pairwise alignment model is designed first, to express the integration procedure of multisource spectral libraries. Second, a statistical intercluster consistency measurement-based benchmark selection strategy for the spectrum pairs crossing two or more spectral libraries is designed, to find out the most suitable spectrum for alignment. Then, a nonnegative background component-based alignment strategy is proposed, to achieve spectral library integration through common background components extracted from the selected benchmark clusters. And finally, a modified spectral super-resolution procedure base on the integrated spectral library is given, to evaluate its effectiveness indirectly. Experimental results with the related spectral super-resolution methods on different datasets demonstrate that, our proposed method can significantly improve the performance of spectral super-resolution in both spatial and spectral domains.

Hyperspectral image classification is mainly based on the spectral information of land covers, but water vapor or Rayleigh scattering will weaken the surface reflectance under the effect of adjacent pixels, and thus lead to the reducing of the discriminative information for the subsequent classification tasks. Atmospheric correction for the weakened bands is one of the most traditional ways to deal with this issue, but as a complete atmospheric correction for both of them is difficult, maybe a systematic exclusion of the severely affected bands base on quantitative evaluation is a better choice. In this paper, an evaluation based weaken band exclusion method for the hyperspectral image classification is proposed, trying to remove the severely affected bands without further atmospheric correction. Specifically, an evaluation model to describe how the water vapor and Rayleigh scattering affect the surface reflectance is constructed, by using the statistical relationship between the radiative transfer model and the band weaken index of spectra among the adjacent pixels. And then, with a simulation experiment, it is shown that water vapor and Rayleigh scattering can really weaken the discriminative information of some specific bands, and the band weaken index can serve as an appropriate index to evaluate the weakening degree of those bands. Finally, on this basis, the total framework of evaluation based weaken band exclusion method is given. The effectiveness and the universality of our proposed method have been verified and compared on four representative tasks of the hyperspectral image classification.

Spectral library based spectral super-resolution is an effective but challenging way to obtain high-spatial hyperspectral images from high-spatial multispectral images. However, the incomplete spectral coverage of spectral response functions makes it impossible to comprehensively sense the spectral information in the imaging model, thus greatly limits the performance of spectral super-resolution. To deal with this problem, a new spectral library based spectral super-resolution method under incomplete spectral coverage conditions is proposed in this paper. More specifically, a strategy for acquiring a typical set of spectra from the spectral library is proposed, trying to provide spectral observations under the incomplete spectral coverage conditions. Secondly, taking the typical set of spectra and the remaining spectral library as a priori, a new spectral super-resolution model is established under sparse and low-rank constraints. And then, the spectral dictionary is optimized utilizing the spectral information supplied by the prior spectral library. Finally, its corresponding coefficient matrix is optimized using the spatial information supplied by the multispectral image and the spectral similarity constraint on the typical spectra. Experimental results using different datasets with different spectral response functions show that, our proposed method outperforms other relative state-of-the-art methods in terms of both spectral reconstruction and spatial preservations.

Spectral library based spectral super-resolution from high-spatial multispectral to hyperspectral image is one of the most efficient ways to obtain high-spatial hyperspectral satellite images, which can be used in various applications. Although most of the published universal spectral libraries can really provide reliable spectral information of the common ground objects, poor region specificity of the universal spectral libraries may limit the accuracy of spectral super-resolution and thus the subsequent applications. To address the above issue, this paper proposes a new spectral super-resolution method for high-spatial multispectral satellite images by using the universal and private jointed spectral library. Specifically, a private spectral library consisting of the spectra of interested ground objects is introduced to the universal spectral library, to form a new joint spectral library, and a new spectral super-resolution model by using the joint spectral library is constructed. Then, band matching between the desired high-spatial hyperspectral image and the joint spectral library will be carried out, to map the joint spectral library into a specific spectral library. After that, contributions of the two universal and private spectral dictionaries will be well balanced by a weighting factor under the sparse representation framework. And finally, the spectral dictionary and its related coefficients will be optimized by the alternating direction method of multipliers (ADMM). Comparison results with the relative state-of-the-art methods shown the superiority of this proposed method, and two typical applications of the tobacco and wheat classification will also be given to evaluate its effectiveness in practical applications.

As a convention, satellites and drones are equipped with sensors of both the visible light spectrum and the infrared (IR) spectrum. However, existing remote sensing object detection methods mostly use RGB images captured by the visible light camera while ignoring IR images. Even for algorithms that take RGB-IR image pairs as input, they may fail to extract all potential features in both spectrums. This letter proposes Multispectral DETR, a remote sensing object detector based on the deformable attention mechanism. To enhance multispectral feature extraction and attention, DropSpectrum and SwitchSpectrum methods are further proposed. DropSpectrum facilitates the extraction of multispectral features by requiring the model to detection some of the targets with only one spectrum. SwitchSpectrum eliminates the level bias caused by the fixed order of RGB-IR feature maps and enhances attention on multispectral features. Experiments on the VEDAI dataset show the state-of-the-art performance of Multispectral DETR and the effectiveness of both DropSpectrum and SwitchSpectrum.

Anomalies in remote sensing images are generally reflected in two aspects of spatial and spectral ones, as for the anomaly detection of sea surface using multispectral or hyperspectral images, spectral information is more important. To this end, a novel spectral anomaly detection method based on dictionary optimization is proposed in this letter. More specifically, the normal scene is first defined to distinguish the anomaly. Then, without any assumption about the distribution of anomaly, a spectral dictionary is formulated and derived theoretically with optimization to express the normal scenes. Using the sparse and low-rank constraints, the alternating direction method of multiplier (ADMM) is employed to solve the above optimization in the spectral domain. Finally, for a given sea-surface image to be detected, the error matrix that cannot be fully expressed by the optimized spectral dictionary is regarded as anomalies. It shows certain generality for various kinds of spectral anomalies on the sea surface. Taking multispectral images obtained by the HY-1C satellite as an example, comparisons with related state-of-the-art methods demonstrate that our proposed method achieves the best anomaly detection performance not only for oil-spill pollution but also for algae pollution.

To generate a high-spatial-resolution hyperspectral (HHS) image from a high-spatial-resolution multispectral (HMS) image, both spatial information and spectral information should be considered simultaneously if we want to build a more accurate mapping from HMS to HHS. To this end, a spectral and spatial jointed spectral super-resolution method is proposed in this letter using an end-to-end learning strategy for each subspace with the cluster-based multibranch backpropagation neural network (BPNN). More specifically, in addition to the spectra similarity, a modified superpixel segmentation is introduced to jointly take spatial contextual information into account, and a new framework with it is given. Comparisons on the Columbia University Automated Vision Environment (CAVE) data set show that our proposed method outperforms other relative state-of-the-art methods more than 0.3 in the root mean squared error (RMSE) and more than 1.0 in the spectral angle mapper (SAM) index. Especially, an exemplary application is demonstrated using the synchronized observation data collected by the multispectral and hyperspectral sensors mounted on the HJ-1A satellite at the same time.

Dictionary learning has been widely used in image representation under the framework of sparse theory. But most of the current dictionary learning strategies can only be used for single-frame image separately, which are insufficient from the perspective of incremental information acquisition and global optimization for the sequential or multiframe satellite images. To this end, this paper proposes an incremental dictionary learning method for multiframe satellite images representation in the spectral domain. The incremental dictionary learning is formulated analytically in the framework of sparse representation with low-rank constraint, as a frame-by-frame gradual optimization process of global and local dictionaries, and their corresponding sparse coefficients with the sequence. Specifically, the global dictionary representing the common spectral information of the sequential frames, is optimized by two adjacent frames gradually. Meanwhile, the local dictionary representing the specific spectral information of each frame, is optimized by the newly added frame itself. In addition, an activity ratio for separating the global dictionary from the local dictionaries, an outlier detection method for initializing the local dictionary are also given, and the alternating direction methods of multipliers (ADMM) is employed to implement the above optimization. Comparison results with the related state-of-the-art methods on different datasets demonstrate that, our proposed method achieves the best representation performance in both spatial and spectral domains, and also helps to improve the performance of dictionary-based tasks using sequential satellite images, such as sea surface anomaly detection.

The performance of deep learning heavily depend on the quantity and quality of training data. But in many fields, well-annotated data are so difficult to collect, which makes the data scale hard to meet the needs of network training. To deal with this issue, a novel data augmentation method using the bitplane information recombination model (termed as BIRD) is proposed in this paper. Considering each bitplane can provide different structural information at different levels of detail, this method divides the internal hierarchical structure of a given image into different bitplanes, and reorganizes them by bitplane extraction, bitplane selection and bitplane recombination, to form an augmented data with different image details. This method can generate up to 62 times of the training data, for a given 8-bits image. In addition, this generalized method is model free, parameter free and easy to combine with various neural networks, without changing the original annotated data. Taking the task of target detection for remotely sensed images and classification for natural images as an example, experimental results on DOTA dataset and CIFAR-100 dataset demonstrated that, our proposed method is not only effective for data augmentation, but also helpful to improve the accuracy of target detection and image classification.