This paper presents a novel self-distillation based deep robust hash for fast remote sensing (RS) image retrieval. Specifically, there are two primary processes in our proposed model: teacher learning (TL) and student learning (SL). Two transformed samples are produced from one sample image through nuanced and signalized transformations, respectively. Transformed samples are fed into both the TL and the SL flows. To reduce discrepancies in the processed samples and guarantee a consistent hash code, the parameters are shared by the two modules during the training stage. Then, a resilient module is employed to enhance the image features in order to ensure more dependable hash code production. Lastly, a three-component loss function is developed to train the entire model. Comprehensive experiments are conducted on two common RS datasets: UCMerced and AID. The experimental results validate that the proposed method has competitive performance against other RS image hashing methods.

The increasing volume of remote sensing (RS) data offers substantial benefits for the extraction and interpretation of features from these scenes. Indeed, the detection of distinguishing features among captured materials and objects is crucial for classification purposes, such as in environmental monitoring applications. In these algorithms, the classes characterized by lower correlation often exhibit more distinct and discernible features, facilitating their differentiation in a straightforward manner. Nevertheless, the rise of Big Data provides a wide range of data acquired through multiple decentralized devices, where its susceptibility to be shared among various users or clients presents challenges in safeguarding privacy. Meanwhile, global features for similar classes are required to be learned for generalization purposes in the classification process. To address this, federated learning (FL) emerges as a privacy efficient decentralized solution. Firstly, in such scenarios, proprietary data is held by individual clients participating in the training of a global model. Secondly, clients may encounter challenges in identifying features that are more distinguishable within the data distributions of other clients. In this study, in order to handle these challenges, a novel methodology is proposed that considers the least correlated classes (LCCs) included in each client data distribution. This strategy exploits the distinctive features between classes, thereby enhancing performance and generalization ability in a secure and private environment.

Remote sensing (RS) imagery provides invaluable insights into characterizing the Earth’s land surface within the scope of Earth observation (EO). Technological advances in capture instrumentation, coupled with the rise in the number of EO missions aimed at data acquisition, have significantly increased the volume of accessible RS data. This abundance of information has alleviated the challenge of insufficient training samples, a common issue in the application of machine learning (ML) techniques. In this context, crowd-sourced data play a crucial role in gathering diverse information from multiple sources, resulting in heterogeneous datasets that enable applications to harness a more comprehensive spatial coverage of the surface. However, the sensitive nature of RS data requires ensuring the privacy of the complete collection. Consequently, federated learning (FL) emerges as a privacy-preserving solution, allowing collaborators to combine such information from decentralized private data collections to build efficient global models. This paper explores the convergence between the FL and RS domains, specifically in developing data classifiers. To this aim, an extensive set of experiments is conducted to analyze the properties and performance of novel FL methodologies. The main emphasis is on evaluating the influence of such heterogeneous and disjoint data among collaborating clients. Moreover, scalability is evaluated for a growing number of clients, and resilience is assessed against Byzantine attacks. Finally, the work concludes with future directions and serves as the opening of a new research avenue for developing efficient RS applications under the FL paradigm. The source code is publicly available at https://github.com/hpc-unex/FLmeetsRS.

Benefitting from superior performance of learning a low-dimensional sparse representation, autoencoders (AEs) are widely applied to hyperspectral unmixing (HU) task, which aims at identifying and quantifying material components in hyperspectral pixels. In this letter, we propose an innovative AE named Transformer-enhanced convolutional neural network (CNN) based on intensive feature (TCN) to improve the performance of HU. Specifically, theTCNis based on a spatial-spectral decomposition fusion mechanism and integrates the local modeling capability of CNN with the global context modeling capability of Transformer. OurTCNconsists of the spectral Swin Transformer block (SSTB), spatial convolutional block (SCB), and the intensive feature fusion block (IFFB). First, the SSTB considers the global spectral context, while the SCB is utilized to extract local spatial features. Then, the IFFB achieves effective integration and complementarity of features from different sources and fully utilizes the intensive features to improve the accuracy and stability of theTCN. Experiments on the dataset demonstrate that the performance of our proposed method significantly outperforms other HU methods.

The widespread availability of remotely sensed datasets establishes a cornerstone for comprehensive image retrieval within the realm of remote sensing (RS). In response, the investigation into hashing-driven retrieval methods garners significance, enabling proficient image acquisition within such extensive data magnitudes. Nevertheless, the used datasets in practical applications are invariably less desirable and with long-tailed distribution. The primary hurdle pertains to the substantial discrepancy in class volumes. Moreover, commonly utilized RS datasets for hashing tasks encompass approximately two–three dozen classes. However, real-world datasets exhibit a randomized number of classes, introducing a challenging variability. This article proposes a new centripetal intensive attention hashing (CIAH) mechanism based on intensive attention features for long-tailed distribution RS image retrieval. Specifically, an intensive attention module (IAM) is adopted to enhance the significant features to facilitate the subsequent generation of representative hash codes. Furthermore, to deal with the inherent imbalance of long-tailed distributed datasets, the utilization of a centripetal loss function is introduced. This endeavor constitutes the inaugural effort toward long-tailed distributed RS image retrieval. In pursuit of this objective, a collection of long-tail datasets is meticulously curated using four widely recognized RS datasets, subsequently disseminated as benchmark datasets. The selected fundamental datasets contain 7, 25, 38, and 45 land-use classes to mimic different real RS datasets. Conducted experiments demonstrate that the proposed methodology attains a performance benchmark that surpasses currently existing methodologies.

In recent years, the rapid development of remote sensing (RS) technology has led to a drastic increase in the availability of RS images. This calls for the need to develop new methods able to effectively and efficiently retrieve the required instances from a massive amount of RS imagery. In retrieval tasks, finding the nearest-neighbor sample of the retrieval query is a fundamental research topic. Exhaustive comparison is the simplest method to accomplish this task. However, due to the involved computational complexity and memory limitations, this solution is no longer feasible in large data retrieval tasks. As an important branch of approximate nearest-neighbor retrieval (NNR), hash algorithms transform high-dimensional data into low-bit expressions (hash codes) with elements of 0 and 1 to reduce storage and computational costs. Hash algorithms aim to preserve the same nearest-neighbor relationship between the learned hash codes and the original data. Existing hash algorithms are divided into two classes: shallow and deep methods. Furthermore, deep hash algorithms can be divided into (semi-) supervised and unsupervised algorithms. In this article, representative hash-based RS image retrieval (HBRSIR) methods are reviewed, studying the application of hashing in other areas of the RS community and introducing available datasets and evaluation metrics for RS image retrieval (RSIR). The performance of representative and cross-modal hashing methods is validated using two common RSIR datasets (UCMerced and AID) and a cross-modal dataset (DSRSID). Prospects of future work summarizing HBRSIR are also provided.

With the recent development of remote sensing technology, large image repositories have been collected. In order to retrieve the desired images of massive remote sensing data sets effectively and efficiently, we propose a novel central cohesion gradual hashing (CCGH) mechanism for remote sensing image retrieval. First, we design a deep hashing model based on ResNet-18 which has a shallow architecture and extracts features of remote sensing imagery effectively and efficiently. Then, we propose a new training model by minimizing a central cohesion loss which guarantees that remote-sensing hash codes are as close to their hash code centers as possible. We also adopt a quantization loss which promotes that outputs are binary values. The combination of both loss functions produces highly discriminative hash codes. Finally, a gradual sign-like function is used to reduce quantization errors. By means of the aforementioned developments, our CCGH achieves state-of-the-art accuracy in the task of remote sensing image retrieval. Extensive experiments are conducted on two public remote sensing image data sets. The obtained results support the fact that our newly developed CCGH is competitive with other existing deep hashing methods.

Hyperspectral images (HSIs) comprise plenty of information in the spatial and spectral domain, which is highly beneficial for performing classification tasks in a very accurate way. Recently, attention mechanisms have been widely used in HSI classification due to their ability to extract relevant spatial and spectral features. Notwithstanding their positive results, most of the attentional strategies usually introduce a significant number of parameters to be trained, making the models more complex and increasing the computational load. In this paper, we develop a new parameter-free attention network for HSI classification. The main advantage of our model is that it does not add parameters to the original network (as opposed to other state-of-the-art approaches), whilst providing higher classification accuracies. Extensive experimental validations and quantitative comparisons are conducted –using different benchmark HSIs– to illustrate these advantages. Code is available on https://github.com/mhaut/Free2Resnet.

Autoencoder-based hyperspectral anomaly detectors have received significant attention. The core of these detectors is to reconstruct backgrounds by optimizing autoencoders so that anomalies can be detected by reconstruction residuals. Nevertheless, existing methods are flawed in two aspects: 1) most of them reconstruct the background along with the anomalies, resulting in undesired performance for large target detection in complex backgrounds; 2) they only focus on the encoder optimization part, ignoring the decoder reconstruction quality of the background. Given the above, this paper proposes a background-guided deformable convolutional autoencoder (DCAE) network with three mutually supportive parts, including encoder, decoder, and background guidance modules. In the encoder, deformable convolution is introduced into regular convolution to build the adaptive spatial feature extractor to fit complex spatial structures, whilst a non-local convolution is introduced to build an external feature extractor to capture global spatial relationships. Further, a mask is designed to filter potential anomalous information, curbing the representation of high-frequency anomalies to focus on widespread backgrounds. In the decoder, a background guidance module (considering the physical meaning of linear reconstruction) is built, guiding the proposed network learning via two strategies. One is initializing the weight of the decoder, and another is adding a loss term. Notably, both the number of output channels of the encoder and the decoder construction are determined by the background guidance module, which creates a bridge between the network design and practical situations. A profound analysis demonstrates the outstanding performance of the proposed method, which outperforms traditional and deep learning methods, proving that the novel designs introduced in the network architecture are extremely effective.

Capsule networks have been a breakthrough in the field of automatic image analysis, opening a new frontier in the art for image classification. Nevertheless, these models were initially designed for RGB images and naively applying these techniques to remote sensing hyperspectral images (HSI) may lead to sub-optimal behaviour, blowing up the number of parameters needed to train the model or not correctly modeling the spectral relations between the different layers of the scene. To overcome this drawback, this work implements a new capsule-based architecture with attention mechanism to improve the HSI data processing. The attention mechanism is applied during the concurrent iterative routing procedure through an inverted dot-product attention.

Hyperspectral unmixing aims at extracting pure spectral signatures and estimating their corresponding abundances at each pixel. Traditional unmixing algorithms consider end-member extraction and abundance estimation as two separate steps, and the completion of abundance estimation requires results from other endmember extraction algorithms. Considering that convolutional neural networks (CNNs) have powerful learning and data fitting capabilities, some techniques based on deep learning (DL) have been proposed in the literature. Most of them only utilize spectral information and neglect spatial information. In addition, existing unmixing methods based on DL usually extract the weight and output of a specific activation layer as endmembers and abundances, respectively. In our work, we exploit 3D convolution to propose a new 3D convolution unmixing network (3DCUN) for hyperspectral unmixing. Two types of real data, i.e., Samson and Jasper, are used to evaluate the performance of our proposed 3DCUN in endmember extraction and abundance estimation. The experimental results reflect that our proposed 3DCUN gets accurate results in estimating endmembers and abundances.

The performance of hyperspectral anomaly detection based on representation models is importantly related to the corresponding dictionary. A good dictionary can optimally model background to detect anomalies. To realize adaptively background reconstruction, this paper constructs global-local dictionaries for collaborative representation detector by using adaptive-shape (SA-CRD). Specifically, robust principal component analysis (RPCA) is used to separate background and anomalies preliminarily. Then adaptive-shape neighbor is adopted to build local dictionaries for robust background region, and the robust background region is clustered to construct a global dictionary for potential anomaly region. Finally, global-local dictionaries are used in the collaborative representation model to finish anomaly detection. Obtained results over two real data sets indicate that the proposed method can improve the accuracy of anomaly detection intensively compared to other state-of-art methods.

Endmember estimation consists of two tasks, that is, determining the number of pure spectral constituents (endmembers) and extracting their spectral signatures. We present a new geometric distance-based method for endmember estimation from hyperspectral images (HSIs), which does not need to know the number of endmembers in advance. Our strategy optimizes the widely used maximum distance analysis (MDA) method from two viewpoints. First, the traditional MDA method performs endmember estimation by computing the maximum distances between any pixel and one specific pixel, line, plane, or affine hull (AH) composed by the endmembers that have been formerly extracted. Instead, our new strategy only requires computing the maximum distance between any pixel and one specific AH. This operation provides a simpler way than MDA to estimate endmembers. Second, our strategy exploits a new distance computation between any pixel and an AH and just needs the normal vector (compared to the traditional MDA method, which uses the normal vector and offset). The new distance computation in our method is much more efficient than that in the traditional MDA method.

Advanced remote-sensing instruments produce massively large scenes from the surface of the earth, with very high spatial resolution and dimensionality. Developing methods for efficiently localizing specific objects in a large-scale scene presents a significant challenge, mainly because of the high computational requirements involved. To tackle this issue, we propose a new hashing for localization (HfL) framework that efficiently searches for specific objects in the large-scale scene. It begins by dividing the scene into a large number of overlapping local patches. A lightweight deep hash model, referred to as a tiny hashing network (THNet), encodes the local patches into hash codes. The Hamming distances between the hash code of an object image, i.e., an image containing the specific class of objects to be localized in the scene, and those of all local patches are computed. Small values of the Hamming distance indicate local patches that are similar to the object image. The positions of these local patches in the large-scale scene reflect the regional locations of the specific objects. The hash codes are binary and do not take up much space, and the Hamming distance carries very low-computational overheads. Further, we exploit a class center loss as the THNet training objective, which can comprehensively manage multiple object classes. These features mean that the HfL framework can localize specific objects very quickly, regardless of the size of the scene. Extensive experiments validate the effectiveness and efficiency of the framework. For instance, HfL can find objects in a remote-sensing image of 19584$\times$19584 pixels in only 4.388 s (on a single RTX2080ti), with remarkable localization results. The source codes and datasets are available athttps://github.com/lrhan/HfL, together providing a baseline for fast localizing objects in a large-scale scene.

Spectral unmixing plays a vital role in hyperspectral image analysis. It mainly consists of two procedures, i.e., endmember extraction and abundance estimation. Although most algorithms for each of the two procedures may exhibit good performance, few studies have been done considering both problems simultaneously. Therefore, hyperspectral unmixing accuracy is normally achieved by exploring all possible combinations of the two types of algorithms, which renders high computational overloads. We propose a novel orthogonal projection framework to conduct fast hyperspectral unmixing. It addresses both endmember extraction and abundance estimation with orthogonal projection endmember (OPE) and orthogonal projection abundance (OPA). Especially, the pixel with the largest orthogonal projection on any pixel is considered to be an endmember. We randomly choose one pixel from the hyperspectral data to compute the orthogonal projections of all pixels and extract the pixel with the largest projection as the first endmember. To avoid extracting the same endmembers, we compute orthogonal projections of all pixels to endmembers that have been previously extracted, and the pixel with the largest projection is considered as the next endmember. In terms of abundance estimation, we also utilize the concept of orthogonal projection and search for a diagonal matrix whose multiplication with the endmember matrix is not only a square matrix but also a diagonal matrix. Then, we exploit some specific matrix operations to estimate the abundance of each endmember at every pixel. We have evaluated the proposed OPE and OPA algorithms on synthetic and real data, and the experimental results have validated their effectiveness and efficiency in hyperspectral unmixing.

Anomaly detection has become an important remote sensing application due to the abundant spectral and spatial information contained in hyperspectral images. Recently, hyperspectral anomaly detection methods based on collaborative representation model have attracted significant attention. Nevertheless, these methods have to face two main challenges: (1) all features (spectral signatures) are constrained to share the same representation coefficient, which ignores the differences among features; (2) existing dictionaries for pixel-by-pixel detection model are usually not reliable. To address these issues, this paper proposes a new relaxed collaborative representation detector for hyperspectral anomaly detection by using a novel non-global dictionary. The proposed detector conducts collaborative representation on each feature dimension of the pixel under test, and simultaneously constrains the coding vectors of different features to be similar. To the best of our knowledge, this is the first time that a detection model is built from each feature dimension. To adjust the contributions of each feature, an adaptive feature weight constrained version of the method is also proposed. The non-global dictionary is constructed by combining the k-nearest neighbor method and an existing global dictionary, which is more reliable and practical than the widely used dual windows dictionary. In addition, this paper also designs a band selection strategy for the proposed method. Experiments on five real datasets indicate that the proposed method suppresses background well and outperforms other classical and state-of-the-art methods.

Land cover classification of remotely sensed images is an extremely important and challenging task. During the last two decades, several methods have been proposed to deal with this problem. In particular, convolutional neural network (CNN)-based methods for land cover classification have enjoyed high popularity due to their strong feature extraction and characterization abilities. However, most CNNs-based methods use relatively small kernels (usually, 3 x 3 pixels in size). Increasing the size of the kernel introduces a lot of parameters and renders considerable computational overloads. To address this issue and allow for the processing of large image datasets, the pyramidal convolution (PyConv) network has been adopted. PyConv network contains several levels of kernels with varying scales and depths, and shows significant improvements in the task of visual recognition. In this paper, we evaluate the performance of the PyConv network on the UCMERCED dataset. Our experimental results reveal that the considered approach exhibits good performance and high efficiency in the task of land cover classification.

We explore self-paced boost learning (SPBL) with superpix-elwise features for hyperspectral image classification (HIC). Firstly, we conduct feature extraction using superpixelwise principal component analysis (SuperPCA), which reduces the dimensionality of hyperspectral images considering the project discrepancy in different homogeneous regions. Secondly, we perform classification by SPBL on the extracted features, where the learning just focuses on the pixels to be classified and does not make their spatial neighbours involved. SPBL embraces the power of self-paced learning on classifying from simple to complex and that of boost learning on classifying in a robust fashion. Our method is not deep learning grounded and the training does not demand high computing resources. The experimental results on two public hyperspectral image datasets demonstrate that our method is competitive with several prominent ones.

Fast and accurate remote sensing image retrieval from large data archives has been an important research topic in the remote sensing research literature. Recently, hashing-based remote sensing image retrieval has attracted extreme attention because of its efficient search capabilities. Especially, deep remote sensing image hashing algorithms have been developed based on convolutional neural networks (CNNs) and have shown effective retrieval performance. However, implementing a deep hashing network tends to be highly expensive in terms of storage space and computing resources to be suitable for on-orbit remote sensing image retrieval, which usually operates on resource-limited devices such as satellites and unmanned aerial vehicles (UAVs). To address this limitation, we propose to hash a deep network that in turn hashes remote sensing images. Specifically, we develop a quantized deep learning to hash (QDLH) framework for large-scale remote sensing image retrieval. The weights and activation functions in the QDLH framework are binarized to low-bit representations, which require comparatively much less storage space and computing resources. The QDLH results in a lightweight deep neural network for effective remote sensing image hashing. We conduct extensive experiments on two public remote sensing image data sets by incorporating several state-of-the-art network architectures into our QDLH methodology for remote sensing image hashing. The experimental results demonstrate that the proposed QDLH is effective in saving hardware resources in terms of both storage and computation. Moreover, superior remote sensing image retrieval performance is also achieved by our QDLH, compared with state-of-the-art deep remote sensing image hashing methods.

We explore the use of graphical generative adversarial networks (Graphical-GAN) for synthesizing remote sensing images. The model is probabilistic graphical based generative adversarial networks (GAN). It pairs a generative network G with a recognition network R. Both of them are adversarially trained with a discriminative network D. Particularly, R is employed to infer the underlying causal relationships among both observed and latent variables from real remote sensing images. The advantages of the Graphical-GAN for synthesizing multiple categories of remote sensing images are two fold. Firstly, it considers the underlying causal relationships and captures the true data distribution of remote sensing images. Secondly, the adversarial learning generates synthetic sensing images that are similar to real ones with slight differences. Our remote sensing image synthesis scheme paves a promising way for remote sensing dataset augmentation, which is an effective means of improving the accuracy of learning models. Experimental results with high Inception Scores (IS) validate the effectiveness of the Graphical-GAN for remote sensing image synthesis.