Several spectral and spatial dictionary-based methods exist for fusing a high-spatial-resolution multispectral image (HR-MSI) with a low-spatial-resolution hyperspectral image (LR-HSI). However, using only one type of dictionary is insufficient to preserve spatial and spectral information simultaneously, while utilizing both dictionaries would increase the computational costs. To address this problem, we propose a fast fusion method (called FFD) for HR-MSIs and LR-HSIs via subpixel-shift decomposition. In this method, through joint optimization of low rank and sparsity within the framework of subpixel shift and sparse representation, an ultimate spectral dictionary is acquired along with its associated coefficients. Specifically, the HR-MSI is decomposed into subimage sequences of the same spatial resolution as the LR-HSI first, to replace the use of spatial dictionaries. Subsequently, a new fusion model is constructed based on this decomposition incorporating the constraints of low rank and sparsity, and especially, a low-rank term is introduced to constrain the spectral consistency along the decomposition direction. Then, the model is theoretically derived by using the alternating direction method of multipliers (ADMM) method, and a joint optimization for the spectral dictionary and its sparse coefficients is obtained. Finally, the desired HR-HSI can be reconstructed by using the above fused subimages through a simple inversed composition. Experimental results on different datasets show that compared with the other related methods, our proposed FFD can achieve an equivalent fusion effect to the best of them in a much shorter time.

High-spatial-resolution multispectral (HM) and low-spatial-resolution hyperspectral (LH) image fusion over the same scene has been intensively studied. However, in practical application scenarios, HM image usually covers a larger area than LH image, thus the same scene-oriented fusion framework can only be used to a limited range of the overlapping area. To solve this problem, a dictionary expansion based incompletely overlapped HM and LH fusion method is proposed here, try to expand the dictionary learned within the overlapping area to the other non-overlapping area, and then to reconstruct an entire high-spatial-resolution hyperspectral (HH) image over the area covered by the whole HM image. Specifically, the above incomplete fusion problem is divided into one fusion problem in the overlapping area and one reconstruction problem over the non-overlapping area separately, where the former is formulated as a traditional spectral dictionary learning based fusion process, and the latter is formulated as a new dictionary expansion based reconstruction process only using the HM image in the framework of sparse and low-rank representation. To ensure the dictionary learned within overlapping area could be precisely expanded to non-overlapping area, a strategy for extracting universal spectra is proposed by using the spectral similarity between overlapping and the non-overlapping area. The expanded spectral dictionary is calculated using the spectral information from the LH image and the co-constraints applied to the representation error in both overlapping and non-overlapping areas, and the corresponding sparse coefficient matrix is calculated using the spatial information from the HM image and a spectral similarity constraint between them. Experimental results on simulated and real datasets with different coverage areas indicate that, our proposed method gives better performance in fusing incompletely overlapped multispectral and hyperspectral images, comparing with other related or tangentially related state-of-the-art methods.

Public spectral library that can provide authentic and reliable spectral information has been widely used in various applications, especially in the spectral library based spectral super-resolution. In general, the joint of multisource spectral libraries from different organizations can provide more varied and richer spectral information, but the differences in collection conditions and equipment between multisource spectra may greatly affect their application effectiveness. To cope with this problem, an integration method of multisource spectral libraries via benchmark alignment for spectral super-resolution is proposed. In this method, a pairwise alignment model is designed first, to express the integration procedure of multisource spectral libraries. Second, a statistical intercluster consistency measurement-based benchmark selection strategy for the spectrum pairs crossing two or more spectral libraries is designed, to find out the most suitable spectrum for alignment. Then, a nonnegative background component-based alignment strategy is proposed, to achieve spectral library integration through common background components extracted from the selected benchmark clusters. And finally, a modified spectral super-resolution procedure base on the integrated spectral library is given, to evaluate its effectiveness indirectly. Experimental results with the related spectral super-resolution methods on different datasets demonstrate that, our proposed method can significantly improve the performance of spectral super-resolution in both spatial and spectral domains.

Hyperspectral image classification is mainly based on the spectral information of land covers, but water vapor or Rayleigh scattering will weaken the surface reflectance under the effect of adjacent pixels, and thus lead to the reducing of the discriminative information for the subsequent classification tasks. Atmospheric correction for the weakened bands is one of the most traditional ways to deal with this issue, but as a complete atmospheric correction for both of them is difficult, maybe a systematic exclusion of the severely affected bands base on quantitative evaluation is a better choice. In this paper, an evaluation based weaken band exclusion method for the hyperspectral image classification is proposed, trying to remove the severely affected bands without further atmospheric correction. Specifically, an evaluation model to describe how the water vapor and Rayleigh scattering affect the surface reflectance is constructed, by using the statistical relationship between the radiative transfer model and the band weaken index of spectra among the adjacent pixels. And then, with a simulation experiment, it is shown that water vapor and Rayleigh scattering can really weaken the discriminative information of some specific bands, and the band weaken index can serve as an appropriate index to evaluate the weakening degree of those bands. Finally, on this basis, the total framework of evaluation based weaken band exclusion method is given. The effectiveness and the universality of our proposed method have been verified and compared on four representative tasks of the hyperspectral image classification.

Spectral library based spectral super-resolution is an effective but challenging way to obtain high-spatial hyperspectral images from high-spatial multispectral images. However, the incomplete spectral coverage of spectral response functions makes it impossible to comprehensively sense the spectral information in the imaging model, thus greatly limits the performance of spectral super-resolution. To deal with this problem, a new spectral library based spectral super-resolution method under incomplete spectral coverage conditions is proposed in this paper. More specifically, a strategy for acquiring a typical set of spectra from the spectral library is proposed, trying to provide spectral observations under the incomplete spectral coverage conditions. Secondly, taking the typical set of spectra and the remaining spectral library as a priori, a new spectral super-resolution model is established under sparse and low-rank constraints. And then, the spectral dictionary is optimized utilizing the spectral information supplied by the prior spectral library. Finally, its corresponding coefficient matrix is optimized using the spatial information supplied by the multispectral image and the spectral similarity constraint on the typical spectra. Experimental results using different datasets with different spectral response functions show that, our proposed method outperforms other relative state-of-the-art methods in terms of both spectral reconstruction and spatial preservations.

Accurate and efficient ship tracking by geosynchronous orbit (GEO) satellites holds great significance for large-scale maritime surveillance. Nevertheless, ship tracking continues to grapple with a multitude of challenges as follows: 1) the targets are small and often obscured by cloud interference, leading to weakened features; 2) the contrasts between the ships and the background are relatively low, complicating the identification and tracking process; and 3) the frame-to-frame relative positioning accuracy is poor, posing difficulties in reflecting the actual movement trends of ships. In response to these challenges, we proposed TS-Track, a novel framework employing multilevel supervision paradigm to improve tracking performance. Initially, this framework restructured the tracking task into three key sub-modules: image enhancement, object tracking, and trajectory adjustment, inherently fostering a unified training protocol that naturally encompasses all components. Subsequently, a trajectory-based frame fusion strategy was proposed, utilizing consecutive three-frame images to enhance target features and produce consistent motion feature patterns; Last but not least, a trajectory adjustment network was developed to correct the position of ships during tracking, resulting in stable tracking trajectories, and reproduce the actual movement trends of ships. The experimental results on GaoFen-4 dataset validated that our method delivered a significant improvement in ship tracking and achieved state-of-the-art (SOTA) performance. Source codes are available athttps://github.com/KTqizhi/KTqizhi.github.io.

Spectral library based spectral super-resolution from high-spatial multispectral to hyperspectral image is one of the most efficient ways to obtain high-spatial hyperspectral satellite images, which can be used in various applications. Although most of the published universal spectral libraries can really provide reliable spectral information of the common ground objects, poor region specificity of the universal spectral libraries may limit the accuracy of spectral super-resolution and thus the subsequent applications. To address the above issue, this paper proposes a new spectral super-resolution method for high-spatial multispectral satellite images by using the universal and private jointed spectral library. Specifically, a private spectral library consisting of the spectra of interested ground objects is introduced to the universal spectral library, to form a new joint spectral library, and a new spectral super-resolution model by using the joint spectral library is constructed. Then, band matching between the desired high-spatial hyperspectral image and the joint spectral library will be carried out, to map the joint spectral library into a specific spectral library. After that, contributions of the two universal and private spectral dictionaries will be well balanced by a weighting factor under the sparse representation framework. And finally, the spectral dictionary and its related coefficients will be optimized by the alternating direction method of multipliers (ADMM). Comparison results with the relative state-of-the-art methods shown the superiority of this proposed method, and two typical applications of the tobacco and wheat classification will also be given to evaluate its effectiveness in practical applications.

Patch-based spatial dictionary has been widely used to fuse a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LMS) image under the framework of sparse representation. However, patch-based dictionary in the spatial domain is not sufficient to preserve spectral information, which may lead to large spectral distortion. To solve this problem, a new spectral dictionary based pansharpening method using subpixel-shift decomposition and joint optimization (termed as PANDA) is proposed. In this method, the model of pansharpening is formulated in a decomposed spectral domain under the sparse and low-rank constraint, as a joint optimization procedure of spectral dictionary and its coefficients. Specifically, a subpixel-shift decomposition is firstly constructed, to decompose the PAN image into a series of subimages with the same spatial resolution of the LMS image. Then, a new imaging model for the pansharpening problem of the LMS image and the decomposed PAN subimages is formulated, with sparse and low-rank constraints. And finally, a joint optimization procedure for the spectral dictionary and its coefficients are theoretically derived, using the spectral information provided by the LMS image and the spatial information provided by the entire PAN subimages, respectively. Experimental results on different datasets show that, the pansharpening performance of the proposed PANDA method outperforms the state-of-the-art methods in both spatial and spectral domains.

Anomalies in remote sensing images are generally reflected in two aspects of spatial and spectral ones, as for the anomaly detection of sea surface using multispectral or hyperspectral images, spectral information is more important. To this end, a novel spectral anomaly detection method based on dictionary optimization is proposed in this letter. More specifically, the normal scene is first defined to distinguish the anomaly. Then, without any assumption about the distribution of anomaly, a spectral dictionary is formulated and derived theoretically with optimization to express the normal scenes. Using the sparse and low-rank constraints, the alternating direction method of multiplier (ADMM) is employed to solve the above optimization in the spectral domain. Finally, for a given sea-surface image to be detected, the error matrix that cannot be fully expressed by the optimized spectral dictionary is regarded as anomalies. It shows certain generality for various kinds of spectral anomalies on the sea surface. Taking multispectral images obtained by the HY-1C satellite as an example, comparisons with related state-of-the-art methods demonstrate that our proposed method achieves the best anomaly detection performance not only for oil-spill pollution but also for algae pollution.

To generate a high-spatial-resolution hyperspectral (HHS) image from a high-spatial-resolution multispectral (HMS) image, both spatial information and spectral information should be considered simultaneously if we want to build a more accurate mapping from HMS to HHS. To this end, a spectral and spatial jointed spectral super-resolution method is proposed in this letter using an end-to-end learning strategy for each subspace with the cluster-based multibranch backpropagation neural network (BPNN). More specifically, in addition to the spectra similarity, a modified superpixel segmentation is introduced to jointly take spatial contextual information into account, and a new framework with it is given. Comparisons on the Columbia University Automated Vision Environment (CAVE) data set show that our proposed method outperforms other relative state-of-the-art methods more than 0.3 in the root mean squared error (RMSE) and more than 1.0 in the spectral angle mapper (SAM) index. Especially, an exemplary application is demonstrated using the synchronized observation data collected by the multispectral and hyperspectral sensors mounted on the HJ-1A satellite at the same time.

Dictionary learning has been widely used in image representation under the framework of sparse theory. But most of the current dictionary learning strategies can only be used for single-frame image separately, which are insufficient from the perspective of incremental information acquisition and global optimization for the sequential or multiframe satellite images. To this end, this paper proposes an incremental dictionary learning method for multiframe satellite images representation in the spectral domain. The incremental dictionary learning is formulated analytically in the framework of sparse representation with low-rank constraint, as a frame-by-frame gradual optimization process of global and local dictionaries, and their corresponding sparse coefficients with the sequence. Specifically, the global dictionary representing the common spectral information of the sequential frames, is optimized by two adjacent frames gradually. Meanwhile, the local dictionary representing the specific spectral information of each frame, is optimized by the newly added frame itself. In addition, an activity ratio for separating the global dictionary from the local dictionaries, an outlier detection method for initializing the local dictionary are also given, and the alternating direction methods of multipliers (ADMM) is employed to implement the above optimization. Comparison results with the related state-of-the-art methods on different datasets demonstrate that, our proposed method achieves the best representation performance in both spatial and spectral domains, and also helps to improve the performance of dictionary-based tasks using sequential satellite images, such as sea surface anomaly detection.

The performance of deep learning heavily depend on the quantity and quality of training data. But in many fields, well-annotated data are so difficult to collect, which makes the data scale hard to meet the needs of network training. To deal with this issue, a novel data augmentation method using the bitplane information recombination model (termed as BIRD) is proposed in this paper. Considering each bitplane can provide different structural information at different levels of detail, this method divides the internal hierarchical structure of a given image into different bitplanes, and reorganizes them by bitplane extraction, bitplane selection and bitplane recombination, to form an augmented data with different image details. This method can generate up to 62 times of the training data, for a given 8-bits image. In addition, this generalized method is model free, parameter free and easy to combine with various neural networks, without changing the original annotated data. Taking the task of target detection for remotely sensed images and classification for natural images as an example, experimental results on DOTA dataset and CIFAR-100 dataset demonstrated that, our proposed method is not only effective for data augmentation, but also helpful to improve the accuracy of target detection and image classification.

Spectral or spatial dictionary has been widely used in fusing low-spatial-resolution hyperspectral (LH) images and high-spatial-resolution multispectral (HM) images. However, only using spectral dictionary is insufficient for preserving spatial information, and vice versa. To address this problem, a new LH and HM image fusion method termed OTD using optimized twin dictionaries is proposed in this paper. The fusion problem of OTD is formulated analytically in the framework of sparse representation, as an optimization of twin spectral-spatial dictionaries and their corresponding sparse coefficients. More specifically, the spectral dictionary representing the generalized spectrums and its spectral sparse coefficients are optimized by utilizing the observed LH and HM images in the spectral domain; and the spatial dictionary representing the spatial information and its spatial sparse coefficients are optimized by modeling the rest of high-frequency information in the spatial domain. In addition, without non-negative constraints, the alternating direction methods of multipliers (ADMM) are employed to implement the above optimization process. Comparison results with the related state-of-the-art fusion methods on various datasets demonstrate that our proposed OTD method achieves a better fusion performance in both spatial and spectral domains.

High-spatial hyperspectral (HH) image reconstruction using both high-spatial multispectral (HM) image and low-spatial hyperspectral (LH) image over the same scene is widely used in many real applications. Nevertheless, the pair of HM image and LH image over the same scene is hard to obtain. To solve this problem, a new HH image reconstruction method using spectral library-based dictionary learning (named as HIRSL) is proposed in this paper, only from one HM image. The above reconstruction problem is formulated in the framework of sparse representation, as an estimation of the band matching matrix, the spectral dictionary, and the sparse coefficients. More specifically, a band matching method is proposed for mapping the common spectral library to a specific spectral library corresponding to the reconstructed HH image in spectral domain. Then, an efficient spectral dictionary learning method is proposed for the construction of spectral dictionary using the matched specific spectral library, which avoids the dependence of the LH image over the same scene. Finally, the sparse coefficients of the HM image with respect to the learned spectral dictionary are estimated using the alternating direction method of multipliers without nonnegative constraint. Comparison results on simulated and real data sets with the relative state-of-the-art methods demonstrate that even only using one HM image, our proposed method achieves a comparable reconstruction quality of high-spatial hyperspectral image both in spatial and spectral domains.

Non-negative Matrix Factorization is the most typical model for hyperspectral image super-resolution. However, the non-negative restriction on the coefficients limited the efficiency of dictionary expression. Facing this problem, a new hyperspectral image super-resolution method based on non-factorization sparse representation and dictionary learning (called NFSRDL) is proposed in this paper. Firstly, an efficient spectral dictionary learning method is specifically adopted for the construction of spectral dictionary using some low spatial resolution hyperspectral images in the same or similar areas. Then, the sparse codes of the high-resolution multi-bands image with respect to the learned spectral dictionary are estimated using the alternating direction method of multipliers (ADMM) without non-negative constrains. Experimental results on different datasets demonstrate that, compared with the related state-of-the-art methods, our method can improve PSNR over 1.3282 and SAM over 0.0476 in the same scene, and PSNR over 3.1207 and SAM over 0.4344 in the similar scenes.