• This work proposes ViT-NTC, a novel Vision Transformer for network traffic classification. • ViT-NTC achieves superior accuracy for VPN and non-VPN service identification with few parameters. • Our experiments show ViT-NTC outperforms CNN and baseline ViT on encrypted and unencrypted data. • The ViT-NTC maintains high accuracy even in low-data training scenarios. • This model enables real-time 6G QoS management and enhanced cybersecurity applications. Previous mobile technologies such as 4G and 5G were not inherently designed for Artificial Intelligence (AI) in mind. However, the development of Beyond 5G (B5G) and 6G key topics like security, AI, energy efficiency, and ubiquitous coverage are being taken into account. The usage of deep learning models for the networking data analysis represent a new challenge for next generation networks. To address these, an AI-native 6G architecture will be proposed to improve the network characteristics in several key aspects: intelligence everywhere, a distributed data infrastructure, zero-touch management, and AI as a service. In this sense, this work presents an analysis of principal deep learning architectures based on Convolutional Neural Networks (CNN) and Transformers, applied to the problem of traffic classification to distinguish between different networking services. The insights derived from the analysis are intended to be applicable to emerging paradigms in 6G networks, such as zero-touch management. In this sense, our main contribution lies in the design and analysis of the ViT-NTC model, a Vision Transformer-based architecture that outperforms traditional CNN and ResNet approaches by delivering significantly higher accuracy and efficiency in classifying VPN and non-VPN network traffic, thus providing a robust and scalable solution for 6G network service differentiation. The experimental results demonstrate that the proposed model achieves higher classification accuracy while reducing computational requirements, thereby enabling the processing of larger input data.

Abstract Geospatial analysis often treats data distributed over spherical surfaces, necessitating specialized statistical methods. Traditional planar Euclidean statistical tools, however, fail to accurately capture directional patterns inherent in such data, resulting in distortions or an oversimplification of directional patterns when applied to curved contexts. This paper introduces QSphericalStats, a QGIS plugin designed to perform robust spherical statistical analyses through the use of an accessible, user-friendly interface. By extending QGIS’s native geospatial capabilities, QSphericalStats enables the analysis of three-dimensional directional data, such as orientations derived from digital elevation models (DEMs). Two case studies were conducted to assess the plugin’s effectiveness in practical scenarios. The first compares LiDAR DEMs and ASTER GDEM datasets, demonstrating the tool’s accuracy in analyzing curved surface data; the second examines cartographic maps with associated 3D information, highlighting its ability to extract and interpret directional trends from complex geospatial datasets. Results from both studies show stable mean direction estimates and moderate concentration parameters, underscoring the plugin’s robustness and the added interpretative value of spherical statistics in geospatial research.

Hyperspectral unmixing addresses the challenge of mixed pixels in hyperspectral images by identifying the number of pure pixels (endmembers), extracting their spectral signatures, and estimating their proportions (abundances) in each pixel composing the scene. Traditional hyperspectral unmixing methods often struggle with scalability and computational efficiency when dealing with gigabyte-scale datasets. In this paper, we propose a distributed parallel geometric distance (DPGD) method for hyperspectral unmixing, exploring the computational power and benefits of distributed parallel processing within a distributed computing framework. The proposed DPGD leverages geometric distance measurements to accurately identify endmembers and estimate their abundances, taking into account the intrinsic similarities within hyperspectral images. This provides a clearer representation of the data structure, leading to improved unmixing accuracy. By using the Spark programming model, the computational workload is efficiently distributed across multiple nodes, significantly reducing processing time. Experimental results on real hyperspectral datasets demonstrate that DPGD scales effectively up to 32 nodes and 290.9 GB of data, achieving competitive accuracy and efficiency compared to state-of-the-art methods. The code is available at https://github.com/ccaadaro/DPDG.

Hyperspectral images capture numerous narrow spectral bands to provide detailed information to identify and locate targets, making them highly suitable for anomaly detection tasks. In recent years, deep learning techniques have demonstrated impressive capabilities and prospects in hyperspectral anomaly detection (HAD), primarily relying on multilayer perceptrons (MLPs) and convolutional neural networks (CNNs) to extract and represent nonlinear features. However, MLPs and CNNs often require deeper network architectures when dealing with complex high-dimensional data, resulting in a constrained generalization and limited representation of features. To address this issue, and inspired by the recent Kolmogorov-Arnold network (KAN), this article introduces a novel asymmetric convolutional autoencoder (AE) network by integrating KAN and CNN, namedKACNet. Specifically, we design a spectral KAN block in the convolutional encoder and a spatial KAN block in the convolutional decoder, to simultaneously enhance the feature extraction and characterization capabilities of the network. Furthermore, to effectively utilize the limited prior information, a weight initialization mechanism based on hierarchical density-based spatial clustering of applications with noise (HDBSCAN) is developed to boost the background recovery. By combining KAN, CNN, and HDBSCAN, the proposed integration enhances the interpretability and reliability of HAD. Extensive experiments are conducted on six public datasets, demonstrating that the KAN poses remarkable performance on background reconstruction, particularly, the proposedKACNetsignificantly outperforms the other state-of-the-art methods.

This paper presents a novel self-distillation based deep robust hash for fast remote sensing (RS) image retrieval. Specifically, there are two primary processes in our proposed model: teacher learning (TL) and student learning (SL). Two transformed samples are produced from one sample image through nuanced and signalized transformations, respectively. Transformed samples are fed into both the TL and the SL flows. To reduce discrepancies in the processed samples and guarantee a consistent hash code, the parameters are shared by the two modules during the training stage. Then, a resilient module is employed to enhance the image features in order to ensure more dependable hash code production. Lastly, a three-component loss function is developed to train the entire model. Comprehensive experiments are conducted on two common RS datasets: UCMerced and AID. The experimental results validate that the proposed method has competitive performance against other RS image hashing methods.

The increasing volume of remote sensing (RS) data offers substantial benefits for the extraction and interpretation of features from these scenes. Indeed, the detection of distinguishing features among captured materials and objects is crucial for classification purposes, such as in environmental monitoring applications. In these algorithms, the classes characterized by lower correlation often exhibit more distinct and discernible features, facilitating their differentiation in a straightforward manner. Nevertheless, the rise of Big Data provides a wide range of data acquired through multiple decentralized devices, where its susceptibility to be shared among various users or clients presents challenges in safeguarding privacy. Meanwhile, global features for similar classes are required to be learned for generalization purposes in the classification process. To address this, federated learning (FL) emerges as a privacy efficient decentralized solution. Firstly, in such scenarios, proprietary data is held by individual clients participating in the training of a global model. Secondly, clients may encounter challenges in identifying features that are more distinguishable within the data distributions of other clients. In this study, in order to handle these challenges, a novel methodology is proposed that considers the least correlated classes (LCCs) included in each client data distribution. This strategy exploits the distinctive features between classes, thereby enhancing performance and generalization ability in a secure and private environment.

Hyperspectral sensors carried by lunar rovers or satellites can effectively invert the mineral abundance of the lunar surface. Due to special environment of the lunar surface and small number of samples, it is a challenge to analyze typical minerals on the lunar surface using hyperspectral images. In this paper, a spectral-spatial diffusion autoencoder unmixing model (SSDiffAU) is proposed for mineral mapping. This is the first time the diffusion model is invoked in the unmixing field. The 3D-CNN is utilized as an encoder to represent deep spectral-spatial information. Additionally, the diffusion model is used to obtain high quality abundance maps, and then the endmember matrix is obtained by the decoder. Finally, the performance of the proposed algorithm was verified using popular unmixing datasets. The hyperspectral data acquired by Chang'e-3 Yutu rover are unmixed to estimate typical mineral spectra and their abundance.

Remote sensing (RS) imagery provides invaluable insights into characterizing the Earth’s land surface within the scope of Earth observation (EO). Technological advances in capture instrumentation, coupled with the rise in the number of EO missions aimed at data acquisition, have significantly increased the volume of accessible RS data. This abundance of information has alleviated the challenge of insufficient training samples, a common issue in the application of machine learning (ML) techniques. In this context, crowd-sourced data play a crucial role in gathering diverse information from multiple sources, resulting in heterogeneous datasets that enable applications to harness a more comprehensive spatial coverage of the surface. However, the sensitive nature of RS data requires ensuring the privacy of the complete collection. Consequently, federated learning (FL) emerges as a privacy-preserving solution, allowing collaborators to combine such information from decentralized private data collections to build efficient global models. This paper explores the convergence between the FL and RS domains, specifically in developing data classifiers. To this aim, an extensive set of experiments is conducted to analyze the properties and performance of novel FL methodologies. The main emphasis is on evaluating the influence of such heterogeneous and disjoint data among collaborating clients. Moreover, scalability is evaluated for a growing number of clients, and resilience is assessed against Byzantine attacks. Finally, the work concludes with future directions and serves as the opening of a new research avenue for developing efficient RS applications under the FL paradigm. The source code is publicly available at https://github.com/hpc-unex/FLmeetsRS.

Benefitting from superior performance of learning a low-dimensional sparse representation, autoencoders (AEs) are widely applied to hyperspectral unmixing (HU) task, which aims at identifying and quantifying material components in hyperspectral pixels. In this letter, we propose an innovative AE named Transformer-enhanced convolutional neural network (CNN) based on intensive feature (TCN) to improve the performance of HU. Specifically, theTCNis based on a spatial-spectral decomposition fusion mechanism and integrates the local modeling capability of CNN with the global context modeling capability of Transformer. OurTCNconsists of the spectral Swin Transformer block (SSTB), spatial convolutional block (SCB), and the intensive feature fusion block (IFFB). First, the SSTB considers the global spectral context, while the SCB is utilized to extract local spatial features. Then, the IFFB achieves effective integration and complementarity of features from different sources and fully utilizes the intensive features to improve the accuracy and stability of theTCN. Experiments on the dataset demonstrate that the performance of our proposed method significantly outperforms other HU methods.

The widespread availability of remotely sensed datasets establishes a cornerstone for comprehensive image retrieval within the realm of remote sensing (RS). In response, the investigation into hashing-driven retrieval methods garners significance, enabling proficient image acquisition within such extensive data magnitudes. Nevertheless, the used datasets in practical applications are invariably less desirable and with long-tailed distribution. The primary hurdle pertains to the substantial discrepancy in class volumes. Moreover, commonly utilized RS datasets for hashing tasks encompass approximately two–three dozen classes. However, real-world datasets exhibit a randomized number of classes, introducing a challenging variability. This article proposes a new centripetal intensive attention hashing (CIAH) mechanism based on intensive attention features for long-tailed distribution RS image retrieval. Specifically, an intensive attention module (IAM) is adopted to enhance the significant features to facilitate the subsequent generation of representative hash codes. Furthermore, to deal with the inherent imbalance of long-tailed distributed datasets, the utilization of a centripetal loss function is introduced. This endeavor constitutes the inaugural effort toward long-tailed distributed RS image retrieval. In pursuit of this objective, a collection of long-tail datasets is meticulously curated using four widely recognized RS datasets, subsequently disseminated as benchmark datasets. The selected fundamental datasets contain 7, 25, 38, and 45 land-use classes to mimic different real RS datasets. Conducted experiments demonstrate that the proposed methodology attains a performance benchmark that surpasses currently existing methodologies.

In recent years, the rapid development of remote sensing (RS) technology has led to a drastic increase in the availability of RS images. This calls for the need to develop new methods able to effectively and efficiently retrieve the required instances from a massive amount of RS imagery. In retrieval tasks, finding the nearest-neighbor sample of the retrieval query is a fundamental research topic. Exhaustive comparison is the simplest method to accomplish this task. However, due to the involved computational complexity and memory limitations, this solution is no longer feasible in large data retrieval tasks. As an important branch of approximate nearest-neighbor retrieval (NNR), hash algorithms transform high-dimensional data into low-bit expressions (hash codes) with elements of 0 and 1 to reduce storage and computational costs. Hash algorithms aim to preserve the same nearest-neighbor relationship between the learned hash codes and the original data. Existing hash algorithms are divided into two classes: shallow and deep methods. Furthermore, deep hash algorithms can be divided into (semi-) supervised and unsupervised algorithms. In this article, representative hash-based RS image retrieval (HBRSIR) methods are reviewed, studying the application of hashing in other areas of the RS community and introducing available datasets and evaluation metrics for RS image retrieval (RSIR). The performance of representative and cross-modal hashing methods is validated using two common RSIR datasets (UCMerced and AID) and a cross-modal dataset (DSRSID). Prospects of future work summarizing HBRSIR are also provided.

Hyperspectral unmixing is a vibrant research field that focuses on the task of decomposing mixed pixels into a collection of pure spectral signatures, known as endmembers, along with their corresponding fractional abundances. Conventional unmixing algorithms often need to combine two techniques, namely endmember extraction and abundance estimation, to accomplish the unmixing task. Recently, deep learning (DL) has succeeded in the field of hyperspectral unmixing due to its strong feature learning and data-fitting capabilities. By extracting the output and weight of a particular layer as abundance maps and endmember signatures, available DL methods can directly unmix hyperspectral images. However, in order to improve the performance of spectral unmixing, such available DL methods frequently employ the results of endmember extraction algorithms –in most cases, the well-known vertex component analysis (VCA)– as the initial weights, which leads to significant limitations in their performance: a) the unmixing results are heavily dependent on the initialization given by VCA, and b) the randomness of VCA is passed to the unmixing network. In this paper, we design a new method called abundance-guided spectral and spatial network (A2SN) which not only skips the weights to extract endmember features directly from the network, but also estimates the abundance maps and reconstructs images directly. In particular, the proposed A2SN employs different kernels to capture spectral and spatial information. We also propose an abundance-guided attention spectral and spatial attention network (A2SAN) for hyperspectral unmixing by integrating attention mechanisms into A2SN. As a result, A2SAN is a completely innovative unmixing method that employs attention and reconstruction directly for hyperspectral unmixing, rather than just as modules for information extraction. Most importantly, both A2SN and A2SAN use a weighted summation of the feature maps to reconstruct the image and increase the noise immunity of the network. Experimental results, conducted on both synthetic and real datasets, demonstrate the effectiveness and superiority of A2SN and A2SAN over state-of-the-art unmixing methods. Our full code is released at https://github.com/xuanwentao/A2SN-and-A2SAN for public evaluation.

Hyperspectral anomaly detection, which aims to search and localize potential targets, is a research area with extensive application prospects and profound implications. In recent years, the emergence of unsupervised and self-supervised deep learning for image reconstruction has provided inspiring solutions for hyperspectral anomaly detection. However, due to sensor-induced and environmental effects, the full-image detection networks inevitably reconstruct anomalies along with the background. Existing detectors indirectly mitigate anomaly reconstruction by imposing constraints on hidden features or loss functions, but they provide unsatisfactory performance in large target detection scenarios. This work straightforwardly addresses this issue from the input source, i.e., introducing the concept of masked autoencoders (MAEs) into fully convolutional networks and further developing a sparse-inspired masked convolutional network (SMCNet) consisting of three mutually supportive components: 1) a hierarchical encoder; 2) a sparse projection layer; and 3) a hierarchical decoder. The encoder employs an adaptive potential anomaly masking strategy, leveraging sparse convolution for extracting multidimensional features of the remaining background. Meanwhile, a sparse-guided projection layer is created by discarding the positional embedding technique to populate the uncoded region and guide the background recovery without introducing anomalies. Finally, the decoder couples the hierarchical structure and a hybrid attention mechanism (local-middle–global and spatial-spectral) to refine the background during image recovery, whereas anomalies in the residual map are highlighted. Extensive experiments using ten typical competitors on six different types of datasets validate the effectiveness and generalization ability of the newly proposed SMCNet method.

Graph convolutional networks (GCNs) have been skillfully employed in hyperspectral image (HSI) classification, exhibiting remarkable performance owing to their unique superiority in handling non-Euclidean graph-structured data. However, the inherent absence of predefined connections between pixels in HSI results in the underutilization of the structural and attribute information of the graph edges. Furthermore, the construction of adjacency matrices for large-scale HSI data imposes a huge computational burden on traditional GCNs. Therefore, in this article, a novel method combining relaxed collaborative representation (RCR) and GCN (RCR-GCN) for hyperspectral classification is proposed. Specifically, RCR is adopted to compute the representation coefficients of each feature, reflecting the similarity and diversity among different sample features. Meanwhile, the representation coefficients are applied as edge attributes in the graph, denoting the weights of the connections between neighboring nodes. After that, GCN is employed to classify the graph nodes. Moreover, an efficient version of the RCR-GCN method is developed to boost the computation, which constructs the graph based on superpixel nodes instead of the pixel nodes by using simple linear iterative clustering (SLIC). Extensive experiments on three HSI image datasets demonstrate that the proposed method outperforms other state-of-the-art methods and achieves more efficiency and feasibility in HSI image classification.

The amount of data needed to effectively train modern deep neural architectures has grown significantly, leading to increased computational requirements. These intensive computations are tackled by the combination of last generation computing resources, such as accelerators, or classic processing units. Nevertheless, gradient communication remains as the major bottleneck, hindering the efficiency notwithstanding the improvements in runtimes obtained through data parallelism strategies. Data parallelism involves all processes in a global exchange of potentially high amount of data, which may impede the achievement of the desired speedup and the elimination of noticeable delays or bottlenecks. As a result, communication latency issues pose a significant challenge that profoundly impacts the performance on distributed platforms. This research presents node-based optimization steps to significantly reduce the gradient exchange between model replicas whilst ensuring model convergence. The proposal serves as a versatile communication scheme, suitable for integration into a wide range of general-purpose deep neural network (DNN) algorithms. The optimization takes into consideration the specific location of each replica within the platform. To demonstrate the effectiveness, different neural network approaches and datasets with disjoint properties are used. In addition, multiple types of applications are considered to demonstrate the robustness and versatility of our proposal. The experimental results show a global training time reduction whilst slightly improving accuracy. Code: https://github.com/mhaut/eDNNcomm.

Cerebrovascular accident (CVA) or stroke is one of the main causes of mortality and morbidity today, causing permanent disabilities. Its early detection helps reduce its effects and its mortality: time is brain. Currently, non-contrast computed tomography (NCCT) continues to be the first-line diagnostic method in stroke emergencies because it is a fast, available, and cost-effective technique that makes it possible to rule out haemorrhage and focus attention on the ischemic origin, that is, due to obstruction to arterial flow. NCCT are quantified using a scoring system called ASPECTS (Alberta Stroke Program Early Computed Tomography Score) according to the affected brain structures. This paper aims to detect in an initial phase those CTs of patients with stroke symptoms that present early alterations in CT density using a binary classifier of CTs without and with stroke, to alert the doctor of their existence. For this, several well-known neural network architectures are implemented in the ImageNet challenges (VGG, NasNet, ResNet and DenseNet), with 3D images, covering the entire brain volume. The training results of these networks are exposed, in which different parameters are tested to obtain maximum performance, which is achieved with a DenseNet3D network that achieves an accuracy of 98% in the training set and 95% in the test set.

With the recent development of remote sensing technology, large image repositories have been collected. In order to retrieve the desired images of massive remote sensing data sets effectively and efficiently, we propose a novel central cohesion gradual hashing (CCGH) mechanism for remote sensing image retrieval. First, we design a deep hashing model based on ResNet-18 which has a shallow architecture and extracts features of remote sensing imagery effectively and efficiently. Then, we propose a new training model by minimizing a central cohesion loss which guarantees that remote-sensing hash codes are as close to their hash code centers as possible. We also adopt a quantization loss which promotes that outputs are binary values. The combination of both loss functions produces highly discriminative hash codes. Finally, a gradual sign-like function is used to reduce quantization errors. By means of the aforementioned developments, our CCGH achieves state-of-the-art accuracy in the task of remote sensing image retrieval. Extensive experiments are conducted on two public remote sensing image data sets. The obtained results support the fact that our newly developed CCGH is competitive with other existing deep hashing methods.

Classifying remotely sensed hyperspectral images (HSIs) became a computationally demanding task given the extensive information contained throughout the spectral dimension. Furthermore, burgeoning data volumes compound inherent computational and store challenges for data processing and classification purposes. Given their distributed processing capabilities, cloud environments have emerged as feasible solutions to handle these hurdles. This encourages the development of innovative distributed classification algorithms that take full advantage of the processing capabilities of such environments. Recently, computational-efficient methods have been implemented to boost network convergence by reducing the required training calculations. This paper develops a novel cloud-based distributed implementation of the Extreme Learning Machine (CC-ELM) algorithm for efficient HSI classification. The proposal implements a fault-tolerant and scalable computing design, whilst avoiding traditional batch-based back-propagation. CC-ELM has been evaluated over state-of-the-art HSI classification benchmarks, yielding promising results and proving the feasibility of cloud environments for large remote sensing and HSI data volumes processing. Code available on: https://github.com/mhaut/scalable-ELM-HSI.

Neural networks (NNs) have gained importance in hyperspectral image (HSI) segmentation for earth observation due to its unparalleled data-driven feature extraction capability. However, in many real-life situations, ground truth is not available, and the performance of unsupervised NNs is still susceptible to enhancement. To overcome this challenge, this letter presents a new loss function to improve the performance of unsupervised HSI segmentation models. The spectral loss function,Sl, which can be included in different models, is based on the purity of the unmixing endmembers and the spectral similarity of the clusters provided by the NN to determine the classes. It is incorporated into a 3D convolutional autoencoder to validate its performance on four standard HSI benchmarks. Furthermore, its performance has been qualitatively examined in a real case study, an oil spill without ground truth. The results show thatSlis a breakthrough in unsupervised HS segmentation, obtaining the best overall performance and highlighting the importance of spectral signatures. Additionally, dimensional reduction is also vital in compacting the spectral information, which facilitates its segmentation. Source code available at [https://github.com/mhaut/HSI-3DSpLoss].

Convolutional models have provided outstanding performance in the analysis of hyperspectral images (HSIs). These architectures are carefully designed to extract intricate information from non-linear features for classification tasks. Notwithstanding their results, model architectures are manually engineered and further optimized for generalized feature extraction. In general terms, deep architectures are time consuming for complex scenarios since they require fine tuning. Neural architecture search (NAS) has emerged as a suitable approach to tackle this shortcoming. In parallel, modern attention-based methods have boosted the recognition of sophisticated features. The search for optimal neural architectures combined with attention procedures motivates the development of this work. This paper develops a new method to automatically design and optimize convolutional neural networks (CNNs) for HSI classification using channel-based attention mechanisms. Specifically, one-dimensional (1D) and spectral-spatial (3D) classifiers are considered to handle the large amount of information contained in HSIs from different perspectives. Furthermore, the proposed AAtt-CNN method meets the requirement to lower the large computational overheads associated with architectural search. It is compared with current state-of-the-art (SOTA) classifiers. Our experiments, conducted using a wide range of HSI images, demonstrate that AAtt-CNN succeeds in finding optimal architectures for classification, leading to SOTA results.

Hyperspectral images (HSIs) comprise plenty of information in the spatial and spectral domain, which is highly beneficial for performing classification tasks in a very accurate way. Recently, attention mechanisms have been widely used in HSI classification due to their ability to extract relevant spatial and spectral features. Notwithstanding their positive results, most of the attentional strategies usually introduce a significant number of parameters to be trained, making the models more complex and increasing the computational load. In this paper, we develop a new parameter-free attention network for HSI classification. The main advantage of our model is that it does not add parameters to the original network (as opposed to other state-of-the-art approaches), whilst providing higher classification accuracies. Extensive experimental validations and quantitative comparisons are conducted –using different benchmark HSIs– to illustrate these advantages. Code is available on https://github.com/mhaut/Free2Resnet.

Autoencoder-based hyperspectral anomaly detectors have received significant attention. The core of these detectors is to reconstruct backgrounds by optimizing autoencoders so that anomalies can be detected by reconstruction residuals. Nevertheless, existing methods are flawed in two aspects: 1) most of them reconstruct the background along with the anomalies, resulting in undesired performance for large target detection in complex backgrounds; 2) they only focus on the encoder optimization part, ignoring the decoder reconstruction quality of the background. Given the above, this paper proposes a background-guided deformable convolutional autoencoder (DCAE) network with three mutually supportive parts, including encoder, decoder, and background guidance modules. In the encoder, deformable convolution is introduced into regular convolution to build the adaptive spatial feature extractor to fit complex spatial structures, whilst a non-local convolution is introduced to build an external feature extractor to capture global spatial relationships. Further, a mask is designed to filter potential anomalous information, curbing the representation of high-frequency anomalies to focus on widespread backgrounds. In the decoder, a background guidance module (considering the physical meaning of linear reconstruction) is built, guiding the proposed network learning via two strategies. One is initializing the weight of the decoder, and another is adding a loss term. Notably, both the number of output channels of the encoder and the decoder construction are determined by the background guidance module, which creates a bridge between the network design and practical situations. A profound analysis demonstrates the outstanding performance of the proposed method, which outperforms traditional and deep learning methods, proving that the novel designs introduced in the network architecture are extremely effective.

Circular data, as a part of directional data engineering, is used in a wide range of fields such as Geology, Biology, Meteorology and Geomatics. It differs from traditional linear data because it is closed and has no beginning or end along the real line, i.e., circular data occurs around a circle, normally measured in degrees. Analyzing directional data, in particular circular data, requires methods that are be available in libraries with a well-known prestige as Python including SciPy, NumPy or SciKit-Learn libs. However, these libraries have a specific area of expertise and they do not combine information in a useful way for two-dimensional data analysis. In this paper, an open-source library has been implemented to be executed by the Python interpreter, called PyCircularStats. Source code: https://github.com/mhaut/pycircularstats

Analyzing directional data, in particular circular data, requires methods that are being available in libraries with a well-known prestige as Python including SciPy, NumPy or SciKit-Learn libs. An open-source library has been implemented to be executed by the Python interpreter, called PyCircularStats. Source code: https://github.com/mhaut/pycircularstats The potential of PyCircularStats is shown with an example of analyzing two-dimensional data using circular statistics. The practical case chosen is the positional accuracy analysis of a satellite image of LandSat-8 in Cáceres, Spain, with 99 control points taken with GNSS systems. In this work, the possibilities of two-dimensional data analysis using circular statistics using the PyCircularStats tool with the results of this case of use is presented.

Applications from Remote Sensing (RS) unveiled unique challenges to Deep Learning (DL) due to the high volume and complexity of their data. On the one hand, deep neural network architectures have the capability to automatically ex-tract informative features from RS data. On the other hand, these models have massive amounts of tunable parameters, re-quiring high computational capabilities. Distributed DL with data parallelism on High-Performance Computing (HPC) sys-tems have proved necessary in dealing with the demands of DL models. Nevertheless, a single HPC system can be al-ready highly heterogeneous and include different computing resources with uneven processing power. In this context, a standard data parallelism strategy does not partition the data efficiently according to the available computing resources. This paper proposes an alternative approach to compute the gradient, which guarantees that the contribution to the gradi-ent calculation is proportional to the processing speed of each DL model's replica. The experimental results are obtained in a heterogeneous HPC system with RS data and demon-strate that the proposed approach provides a significant training speed up and gain in the global accuracy compared to one of the state-of-the-art distributed DL framework.

Capsule networks have been a breakthrough in the field of automatic image analysis, opening a new frontier in the art for image classification. Nevertheless, these models were initially designed for RGB images and naively applying these techniques to remote sensing hyperspectral images (HSI) may lead to sub-optimal behaviour, blowing up the number of parameters needed to train the model or not correctly modeling the spectral relations between the different layers of the scene. To overcome this drawback, this work implements a new capsule-based architecture with attention mechanism to improve the HSI data processing. The attention mechanism is applied during the concurrent iterative routing procedure through an inverted dot-product attention.

Hyperspectral unmixing aims at extracting pure spectral signatures and estimating their corresponding abundances at each pixel. Traditional unmixing algorithms consider end-member extraction and abundance estimation as two separate steps, and the completion of abundance estimation requires results from other endmember extraction algorithms. Considering that convolutional neural networks (CNNs) have powerful learning and data fitting capabilities, some techniques based on deep learning (DL) have been proposed in the literature. Most of them only utilize spectral information and neglect spatial information. In addition, existing unmixing methods based on DL usually extract the weight and output of a specific activation layer as endmembers and abundances, respectively. In our work, we exploit 3D convolution to propose a new 3D convolution unmixing network (3DCUN) for hyperspectral unmixing. Two types of real data, i.e., Samson and Jasper, are used to evaluate the performance of our proposed 3DCUN in endmember extraction and abundance estimation. The experimental results reflect that our proposed 3DCUN gets accurate results in estimating endmembers and abundances.

The performance of hyperspectral anomaly detection based on representation models is importantly related to the corresponding dictionary. A good dictionary can optimally model background to detect anomalies. To realize adaptively background reconstruction, this paper constructs global-local dictionaries for collaborative representation detector by using adaptive-shape (SA-CRD). Specifically, robust principal component analysis (RPCA) is used to separate background and anomalies preliminarily. Then adaptive-shape neighbor is adopted to build local dictionaries for robust background region, and the robust background region is clustered to construct a global dictionary for potential anomaly region. Finally, global-local dictionaries are used in the collaborative representation model to finish anomaly detection. Obtained results over two real data sets indicate that the proposed method can improve the accuracy of anomaly detection intensively compared to other state-of-art methods.

Convolutional neural networks (CNNs) are relevant tools for remote sensing data processing in the last few years. Kernels process and integrate the spatial information of remotely sensed hyperspectral images (HSIs) accurately enough, so as to reduce the noise and spectral variations present in the data. Despite the great efficiency of the CNNs with HSI data, kernels need to be composed of a large number of layers and parameters to be able to obtain good performance, forcing the model to process a large amount of information from the datasets and making the model prone to overfitting, due to the limited number of labels in some cases. On the other hand, with that huge amount of data to consume, a large number of kernels are needed, making the model less efficient due to computational complexity. To overcome these challenges, this letter presents a new technique to reduce the computational cost and increase the model accuracy [based on adaptable rectangular convolutions (ARCs)], leading to a substantial reduction of the number of parameters and improving the model, in order to achieve better results in the context of HSI semantic segmentation. Thus, the model learns through the convolution the dimensions and offsets of the kernel of this Adaptable layer, performing the average operation based on integral image works to achieve better results with fewer parameters, reducing the risk of overfitting and computational cost. The source code can be found in the repository available at the link. [https://github.com/jlgs96/segHSI]

Convolutional neural networks (CNNs) have proven to be very efficient for the analysis of remote sensing (RS) images. Due to the inherent complexity of extracting features from these images, along with the increasing amount of data to be processed (and the diversity of applications), there is a clear tendency to develop and employ increasingly deep and complex CNNs. In this regard, graphics processing units (GPUs) are frequently used to optimize their execution, both for the training and inference stages, optimizing the performance of neural models through their many-core architecture. Hence, the efficient use of the GPU resources should be at the core of optimizations. This letter analyzes the possibilities of using a new family of CNNs, denoted as TResNets, to provide an efficient solution to the RS scene classification problem. Moreover, the considered models have been combined with mixed precision to enhance their training performance. Our experimental results, conducted over three publicly available RS data sets, show that the proposed networks achieve better accuracy and more efficient use of GPU resources than other state-of-the-art networks. Source code is available athttps://github.com/mhaut/GPUfriendlyRS.

Land-cover classification methods are based on the processing of large image volumes to accurately extract representative features. Particularly, convolutional models provide notable characterization properties for image classification tasks. Distributed learning mechanisms on high performance computing platforms have been proposed to speed up the processing, whilst achieving an efficient feature extraction. High performance computing platforms are commonly composed of a combination of CPUs and GPUs, with different computational capabilities. As a result, current homogeneous workload distribution techniques for deep learning become obsolete due to their inefficient use of computational resources. To address this, new computational balancing proposals, such as heterogeneous data parallelism, have been implemented. Nevertheless, these techniques should be improved to handle the peculiarities of working with heterogeneous data workloads in the training of distributed deep learning models. The objective of handling heterogeneous workloads for current platforms motivates the development of this work. This paper proposes an innovative heterogeneous gradient calculation applied to land-cover classification tasks through convolutional models, considering the data amount assigned to each device in the platform whilst maintaining the acceleration. Extensive experimentation has been conducted on multiple datasets, considering different deep models on heterogeneous platforms to demonstrate the performance of the proposed methodology.

Land-cover information is of paramount importance in a wide range of environmental and socioeconomic applications. Deep learning (DL) provides a large variety of potential models for extracting useful information from raw images. However, remote sensing image (RSI) classification remains a challenging goal due to the intrinsic features of the data, such as the high sample variability and lack of labeled data. This provides a challenge to the reliability of deep classifiers. In particular, convolution-based models are greatly affected by overfitting and vanishing gradient problems. To overcome these drawbacks, this letter presents a new attention-based architecture, including attention modular blocks. These blocks divide their input feature maps into several groups and split them along the channel dimension and then combine them to create an attention mask encoding global contextual information. The mask is applied to obtain a refined feature representation, strengthening those features that affect most significantly the classification and attenuating the rest. Our new method reduces significantly the number of trainable parameters. Our results, obtained using several widely used RSIs, demonstrate that the new method exhibits higher classification performance when compared to several state-of-the-art methods.

Endmember estimation consists of two tasks, that is, determining the number of pure spectral constituents (endmembers) and extracting their spectral signatures. We present a new geometric distance-based method for endmember estimation from hyperspectral images (HSIs), which does not need to know the number of endmembers in advance. Our strategy optimizes the widely used maximum distance analysis (MDA) method from two viewpoints. First, the traditional MDA method performs endmember estimation by computing the maximum distances between any pixel and one specific pixel, line, plane, or affine hull (AH) composed by the endmembers that have been formerly extracted. Instead, our new strategy only requires computing the maximum distance between any pixel and one specific AH. This operation provides a simpler way than MDA to estimate endmembers. Second, our strategy exploits a new distance computation between any pixel and an AH and just needs the normal vector (compared to the traditional MDA method, which uses the normal vector and offset). The new distance computation in our method is much more efficient than that in the traditional MDA method.

The profound impact of deep learning and particularly of convolutional neural networks (CNNs) in automatic image processing has been decisive for the progress and evolution of remote sensing (RS) hyperspectral imaging (HSI) processing. Indeed, CNNs have stated themselves as the current state of the art, reaching unparalleled results in HSI classification. However, most CNNs were designed for RGB images, and their direct application to HSI data analysis could lead to nonoptimal solutions. Moreover, CNNs perform classification based on the identification of specific features, neglecting the spatial relationships between different features (i.e., their arrangement) due to pooling techniques. The capsule network (CapsNet) architecture is an attempt to overcome this drawback by nesting several neural layers within a capsule, connected by dynamic routing, both to identify not only the presence of a feature but also its instantiation parameters and to learn the relationships between different features. Although this mechanism improves the data representations, enhancing the classification of HSI data, it still acts as a black box, without control of the most relevant features for classification purposes. Indeed, important features could be discriminated against. In this article, a new multiple attention-guided CapsNet is proposed to improve feature processing for RS-HSIs’ classification, both to improve computational efficiency (in terms of parameters) and increase accuracy. Hence, the most representative visual parts of the images are identified using a detailed feature extractor coupled with attention mechanisms. Extensive experimental results have been obtained on five real datasets, demonstrating the great potential of the proposed method compared to other state-of-the-art classifiers.

Recently, convolutional neural networks (CNNs) have exhibited commendable performance for hyperspectral image (HSI) classification. Generally, an important number of samples are needed for each class to properly train CNNs. However, existing HSI data sets suffer from a significant class imbalance problem, where many classes do not have enough samples to characterize the spectral information. The performance of existing CNN models is biased toward the majority classes, which possess more samples for the training. This article addresses this issue of imbalanced data in HSI classification. In particular, a new3D-HyperGAMOmodel is proposed, which uses generative adversarial minority oversampling. The proposed3D-HyperGAMOautomatically generates more samples for minority classes at training time, using the existing samples of that class. The samples are generated in the form of a 3-D hyperspectral patch. A different classifier from the generator and the discriminator is used in the3D-HyperGAMOmodel, which is trained using both original and generated samples to determine the classes of newly generated samples to which they actually belong. The generated data are combined classwise with the original training data set to learn the network parameters of the class. Finally, the trained 3-D classifier network validates the performance of the model using the test set. Four benchmark HSI data sets, namely, Indian Pines (IP), Kennedy Space Center (KSC), University of Pavia (UP), and Botswana (BW), have been considered in our experiments. The proposed model shows outstanding data generation ability during the training, which significantly improves the classification performance over the considered data sets. The source code is available publicly athttps://github.com/mhaut/3D-HyperGAMO.

Spectral unmixing plays a vital role in hyperspectral image analysis. It mainly consists of two procedures, i.e., endmember extraction and abundance estimation. Although most algorithms for each of the two procedures may exhibit good performance, few studies have been done considering both problems simultaneously. Therefore, hyperspectral unmixing accuracy is normally achieved by exploring all possible combinations of the two types of algorithms, which renders high computational overloads. We propose a novel orthogonal projection framework to conduct fast hyperspectral unmixing. It addresses both endmember extraction and abundance estimation with orthogonal projection endmember (OPE) and orthogonal projection abundance (OPA). Especially, the pixel with the largest orthogonal projection on any pixel is considered to be an endmember. We randomly choose one pixel from the hyperspectral data to compute the orthogonal projections of all pixels and extract the pixel with the largest projection as the first endmember. To avoid extracting the same endmembers, we compute orthogonal projections of all pixels to endmembers that have been previously extracted, and the pixel with the largest projection is considered as the next endmember. In terms of abundance estimation, we also utilize the concept of orthogonal projection and search for a diagonal matrix whose multiplication with the endmember matrix is not only a square matrix but also a diagonal matrix. Then, we exploit some specific matrix operations to estimate the abundance of each endmember at every pixel. We have evaluated the proposed OPE and OPA algorithms on synthetic and real data, and the experimental results have validated their effectiveness and efficiency in hyperspectral unmixing.

Anomaly detection has become an important remote sensing application due to the abundant spectral and spatial information contained in hyperspectral images. Recently, hyperspectral anomaly detection methods based on collaborative representation model have attracted significant attention. Nevertheless, these methods have to face two main challenges: (1) all features (spectral signatures) are constrained to share the same representation coefficient, which ignores the differences among features; (2) existing dictionaries for pixel-by-pixel detection model are usually not reliable. To address these issues, this paper proposes a new relaxed collaborative representation detector for hyperspectral anomaly detection by using a novel non-global dictionary. The proposed detector conducts collaborative representation on each feature dimension of the pixel under test, and simultaneously constrains the coding vectors of different features to be similar. To the best of our knowledge, this is the first time that a detection model is built from each feature dimension. To adjust the contributions of each feature, an adaptive feature weight constrained version of the method is also proposed. The non-global dictionary is constructed by combining the k-nearest neighbor method and an existing global dictionary, which is more reliable and practical than the widely used dual windows dictionary. In addition, this paper also designs a band selection strategy for the proposed method. Experiments on five real datasets indicate that the proposed method suppresses background well and outperforms other classical and state-of-the-art methods.

Abstract Nowadays, data processing applications based on neural networks cope with the growth in the amount of data to be processed and with the increase in both the depth and complexity of the neural networks architectures, and hence in the number of parameters to be learned. High-performance computing platforms are provided with fast computing resources, including multi-core processors and graphical processing units, to manage such computational burden of deep neural network applications. A common optimization technique is to distribute the workload between the processes deployed on the resources of the platform. This approach is known as data-parallelism. Each process, known as replica, trains its own copy of the model on a disjoint data partition. Nevertheless, the heterogeneity of the computational resources composing the platform requires to unevenly distribute the workload between the replicas according to its computational capabilities, to optimize the overall execution performance. Since the amount of data to be processed is different in each replica, the influence of the gradients computed by the replicas in the global parameter updating should be different. This work proposes a modification of the gradient computation method that considers the different speeds of the replicas, and hence, its amount of data assigned. The experimental results have been conducted on heterogeneous high-performance computing platforms for a wide range of models and datasets, showing an improvement in the final accuracy with respect to current techniques, with a comparable performance.

This paper proposes a new recommendation system in the context of the HOPE project with the aim of providing medical bibliographic references in a simple, up-to-date and immediate way. In addition, these references are catalogued according to the patient information and symptoms, offering a ranking mechanism which sorts them from most interesting to least relevant ones according to the feedback provided by health professionals. The proposed system has been extensively trained and validated with a set of widely used machine learning models, particularly Random Forest (RF), Multinomial Logistic Regression (MLR) and Support Vector Machines (SVMs). The results obtained over real medical data from HOPE project are quite promising, exhibiting a high precision. In particular, RF is the algorithm which the best behavior with a 89.9% of precision. It is closely followed by the SVM, which reaches great results with a 89.4% of precision, performing quite accurately with false negative cases.

Despite its great potential in a wide range of human activities, hyperspectral remote sensing imaging (HSI) exhibits several challenges that prevent full exploitation of its data. In particular, land-cover classification based on HSI data suffers significant degradation due to problematic data variability. Convolutional Neural Networks (CNNs) ability to extract spectral-spatial features has enabled the development of powerful classifiers, which achieve not yet seen accuracy results. To enhance the feature extraction procedure, this paper presents a novel HSI-CNN model (DKDCNet) which combines adaptive deforming kernels (DK) and convolutions (DC) with the aim of pinpointing the effective receptive field (ERF) on the challenging input data. Experimental results on the University of Houston benchmark show that DKDCNet is able to obtain a more accurate classification than traditional strategies with similar computational cost for HSI classification. Source code: https://github.com/mhaut/DKDCNet.

Land cover classification of remotely sensed images is an extremely important and challenging task. During the last two decades, several methods have been proposed to deal with this problem. In particular, convolutional neural network (CNN)-based methods for land cover classification have enjoyed high popularity due to their strong feature extraction and characterization abilities. However, most CNNs-based methods use relatively small kernels (usually, 3 x 3 pixels in size). Increasing the size of the kernel introduces a lot of parameters and renders considerable computational overloads. To address this issue and allow for the processing of large image datasets, the pyramidal convolution (PyConv) network has been adopted. PyConv network contains several levels of kernels with varying scales and depths, and shows significant improvements in the task of visual recognition. In this paper, we evaluate the performance of the PyConv network on the UCMERCED dataset. Our experimental results reveal that the considered approach exhibits good performance and high efficiency in the task of land cover classification.

Convolutional neural networks (CNNs) have demonstrated excellent performance in hyperspectral image (HSI) classification. However, tuning some critical hyperparameters of a CNN-such as the receptive field (RF) size-presents a major challenge due to the presence of features with different scales in HSIs. Contrary to the conventional design of CNNs, which fixes the RF size, it has been proven that the RF size is modulated by the stimulus and hence, depends on the scene being considered. Such a dilemma has been rarely considered in CNN design. In this letter, a new multibranch selective kernel network (MSKNet) is introduced, in which the input image is convolved using different RF sizes to create multiple branches so that the effect of each branch is adjusted by an attention mechanism according to the input contrast. As a result, our newly developed MSKNet is capable of modeling different scales. Our experimental results, conducted on three widely used HSIs, reveal that the MSKNet can outperform state-of-the-art CNNs in the context of HSI classification problems. The source code of our newly developed MSKNet is available from: https://github.com/mhaut/MSKNet-HSI.

High-resolution digital surface models (DSMs) provide valuable height information about the Earth's surface, which can be successfully combined with other types of remotely sensed data in a wide range of applications. However, the acquisition of DSMs with high spatial resolution is extremely time-consuming and expensive with their estimation from a single optical image being an ill-possed problem. To overcome these limitations, this letter presents a new unpaired approach to obtain DSMs from optical images using deep learning techniques. Specifically, our new deep neural model is based on variational autoencoders (VAEs) and generative adversarial networks (GANs) to perform image-to-image translation, obtaining DSMs from optical images. Our newly proposed method has been tested in terms of photographic interpretation, reconstruction error, and classification accuracy using three well-known remotely sensed data sets with very high spatial resolution (obtained over Potsdam, Vaihingen, and Stockholm). Our experimental results demonstrate that the proposed approach obtains satisfactory reconstruction rates that allow enhancing the classification results for these images. The source code of our method is available from: https://github.com/mhaut/UIMG2DSM.

As a newly emerging technology, deep learning (DL) is a very promising field in big data applications. Remote sensing often involves huge data volumes obtained daily by numerous in-orbit satellites. This makes it a perfect target area for data-driven applications. Nowadays, technological advances in terms of software and hardware have a noticeable impact on Earth observation applications, more specifically in remote sensing techniques and procedures, allowing for the acquisition of data sets with greater quality at higher acquisition ratios. This results in the collection of huge amounts of remotely sensed data, characterized by their large spatial resolution (in terms of the number of pixels per scene), and very high spectral dimensionality, with hundreds or even thousands of spectral bands. As a result, remote sensing instruments on spaceborne and airborne platforms are now generating data cubes with extremely high dimensionality, imposing several restrictions in terms of both processing runtimes and storage capacity. In this article, we provide a comprehensive review of the state of the art in DL for remote sensing data interpretation, analyzing the strengths and weaknesses of the most widely used techniques in the literature, as well as an exhaustive description of their parallel and distributed implementations (with a particular focus on those conducted using cloud computing systems). We also provide quantitative results, offering an assessment of a DL technique in a specific case study (source code available: https://github.com/mhaut/cloud-dnn-HSI). This article concludes with some remarks and hints about future challenges in the application of DL techniques to distributed remote sensing data interpretation problems. We emphasize the role of the cloud in providing a powerful architecture that is now able to manage vast amounts of remotely sensed data due to its implementation simplicity, low cost, and high efficiency compared to other parallel and distributed architectures, such as grid computing or dedicated clusters.

Hyperspectral imaging (HSI) is a competitive remote sensing technique in several fields, from Earth observation to health, robotic vision, and quality control. Each HSI scene contains hundreds of (narrow) contiguous spectral bands. The amount of data generated by HSI devices is often both a solution and a problem for a given application. Extracting information from HSI data cubes is a complex and computationally demanding problem. To tackle this challenge, convolutional neural networks (CNNs) have been widely applied to HSI classification. Despite their success, CNNs are computationally demanding algorithms with high memory requirements due to their large number of internal parameters. The recent interest in using HSI devices in mobile and embedded systems for air and spaceborne platforms turned the attention to computationally lightweight CNN architectures with good classification accuracy. In this article, we present a contribution in that direction. The proposed method combines the ghost-module architecture with a CNN-based HSI classifier to reduce the computational cost and, simultaneously, achieves an efficient classification method with high performance. Our new method is evaluated against nine standard HSI classifiers, and five improved deep-CNN architectures, over five commonly used HSI data sets for algorithm benchmarking. Conducted experiments show that the proposed method exhibits similar or better performance than the other classifiers, achieving top values in the considered performance metrics—even for very limited training sets—and, most importantly, with a fraction of the computational cost. Our novel approach for HSI classification is a strong candidate for implementation on systems with limited computational resources.

Convolutional neural networks (CNNs) have proven to be a powerful tool for the classification of hyperspectral images (HSIs). The CNN kernels are able to naturally include spatial information to smooth out the spectral variability and the noise present in HSI data. However, these kernels are composed of a large number of learning parameters that must be correctly adjusted to achieve good performance. This forces the model to consume a large amount of training data, being prone to overfitting when limited labeled samples are available. In addition, the execution of kernels is computationally very expensive, increasing quadratically with respect to the size of the convolution filter. This significantly reduces the performance of the model. To overcome the aforementioned limitations, this work presents a new few-parameter CNN (based on shift operations) for HSI classification that dramatically reduces both the number of parameters and the computational complexity of the model in terms of floating-point operations (FLOPs). The operational module combines a shift kernel (which adjusts the input data in particular directions without involving any parameters nor FLOPs) with pointwise convolutions that perform the feature extraction stage. The newly developed shift-based CNN has been employed to conduct HSI classification over five widely used and challenging data sets, achieving very promising results in terms of computational performance and classification accuracy.

Hyperspectral imaging (HSI) gathers hundreds of images along the electromagnetic spectrum for the same area on the surface of the Earth, collecting a rich amount of spatial and spectral information. Deep learning classifiers have achieved significantly high precision results when analyzing HSI data. In particular, capsule networks (CapsNets) can provide robust classification results, overcoming the limitations of traditional convolutional neural networks (CNNs) by enriching the feature presentation capability and applying dynamic routing mechanisms. As a result, CapsNets are now widely regarded as the state-of-the-art within deep learning field. However, as it is the case for CNNs, the performance of CapsNets strongly depends on the quantity and quality of the available training samples, which in HSI tends to be scarce and noisy. Moreover, obtaining labeled data is expensive and time-consuming, and the high dimensionality of HSI data makes it difficult to accurately design classifiers based on limited training samples. This is mainly due to the strong intra-class variability present in the HSI data. Active learning (AL) can alleviate the aforementioned problems by selecting a small set of highly-representative labeled samples from a pool of unlabeled data, in iterative fashion. This paper presents a new AL-based approach for HSI data classification that integrates the spectral and the spatial information contained in the HSI data and enhances the performance of CapsNets when very limited training samples are available. Code: https://github.com/mhaut/AL-CapsNet-HSI.

Deep neural networks (DNNs) exhibit great performance in the task of hyperspectral image (HSI) classification. However, these models are usually overparameterized and require large amounts of training data in order to properly avoid the curse of dimensionality and the variability of spectral signatures, thus suffering from overfitting problems when very few training samples are available, due to poor generalization ability in this particular case. The traditional regularization dropout (DO) strategy has been shown to be effective in fully connected DNNs but not in convolutional-based ones. This is mainly due to the way these architectures manage the spatial information. In this letter, we introduce a new approach to improve the generalization of convolutional-based models for HSI classification. Specifically, we develop a neighboring region DO technique that selectively cuts off certain neighboring outputs, creating spatial dropped regions. Our experimental results with two well-known HSIs reveal that the newly proposed method helps to achieve better classification accuracy than the traditional DO strategy, with a low computational cost.

Denoising is a common preprocessing step prior to the analysis and interpretation of hyperspectral images (HSIs). However, the vast majority of methods typically adopted for HSI denoising exploit architectures originally developed for grayscale or RGB images, exhibiting limitations when processing high-dimensional HSI data cubes. In particular, traditional methods do not take into account the high spectral correlation between adjacent bands in HSIs, which leads to unsatisfactory denoising performance as the rich spectral information present in HSIs is not fully exploited. To overcome this limitation, this article considers deep learning models-such as convolutional neural networks (CNNs)-to perform spectral-spatial HSI denoising. The proposed model, called HSI single denoising CNN (HSI-SDeCNN), efficiently takes into consideration both the spatial and spectral information contained in HSIs. Experimental results on both synthetic and real data demonstrate that the proposed HSI-SDeCNN outperforms other state-of-the-art HSI denoising methods. Source code: https://github.com/mhaut/HSI-SDeCNN.

Advances in deep learning (DL) have allowed for the development of more complex and powerful neural architectures. The adoption of deep convolutional-based architectures with residual learning [residual networks (ResNets)] has reached the state-of-the-art performance in hyperspectral image (HSI) classification. Traditionally, ResNets have been considered as stacks of discrete layers, where each one obtains a hidden state of the input data. This formulation must deal with very deep networks, which suffer from an important data degradation as they become deeper. Moreover, these complex models exhibit significant requirements in terms of memory due to the amount of parameters that need to be fine tuned. This leads to inadequate generalization and loss of accuracy. In order to address these issues, this article redesigns the ResNet as a continuous-time evolving model, where hidden representations (or states) are obtained with respect to time (understood as the depth of the network) through the evaluation of an ordinary differential equation (ODE), which is combined with a deep neural architecture. Our experimental results, conducted with four well-known HSI data sets, indicate that redefining deep networks as continuous systems through ODEs offers flexibility when processing and classifying these kinds of remotely sensed data, achieving significant performance even when a very few training samples are available.

The probabilistic Latent Semantic Analysis (pLSA) model has recently shown a great potential to uncover highly descriptive semantic features from limited amounts of remote sensing data. Nonetheless, the high computational cost of this algorithm often constraints its operational application for land cover categorization tasks. In this scenario, this paper presents an Open Multi-Processing (OpenMP) implementation of the pLSA algorithm for unsupervised Synthetic Aperture Radar (SAR) and Multi-Spectral Imaging (MSI) image categorization. The experimental results suggest that multi-core systems are an important architecture for the efficient processing of both SAR and MSI datasets. Specifically, the proposed approach is able to cover a real scenario exhibiting good results in both accuracy and performance terms.

This work proposes a new collaborative active and semi-supervised learning approach, named accessibility-free active learning (AFAL), for hyperspectral imaging classification. The proposed approach aims to tackle an existing problem in traditional active learning methods, that is, the fact that some selected samples are not accessible by oracles for assigning them pseudo labels, i.e., confident predictions for the classifier. The proposal specifically addresses this problem using superpixels in a self-training context. Specifically, AFAL first generates a set of candidates locally around the labeled pixels and then expands them to other subregions via a density peak-based augmentation strategy, in order to guarantee the confidence of pseudo labels. Our experimental results, obtained on two real and well-used hyperspectral images, reveal that the proposed scheme can lead to state-of-the-art performance.

Deep neural networks (DNNs) have revolutionized the way remotely sensed hyperspectral image (HSI) data are managed and processed. For instance, residual networks (ResNets) have achieved high classification accuracy by applying sequential transformations (layer by layer) on the input HSI data, obtaining highly discriminative data representations. However, these models are quite complex, with significant requirements in terms of memory resulting from the large number of parameters that they need to learn, which also leads to potential overfitting issues. In this work, we specifically address the aforementioned problem by re-interpreting a DNN (the ResNet) as a continuous transformation, instead of the traditional (discrete) step-by-step approach. To achieve this, we combine ordinary differential equations (ODEs) with DNN architectures for the first time in the HSI data classification literature. This allows us to perform remotely sensed HSI data classification in an efficient way in terms of number of parameters. Our experimental results, conducted using two well-known HSI data sets, indicate that the inclusion of ODEs in the architecture of DNNs offers significant advantages when processing and classifying this kind of high-dimensional data, achieving better performance even with less training data.

Convolutional neural networks have emerged as an excellent tool for remotely sensed hyperspectral image (HSI) classification. Nonetheless, the high computational complexity and energy requirements of these models typically limit their application in on-board remote sensing scenarios. In this context, low-power consumption architectures are promising platforms that may provide acceptable on-board computing capabilities to achieve satisfactory classification results with reduced energy demand. For instance, the new NVIDIA Jetson Tegra TX2 device is an efficient solution for on-board processing applications using deep-learning (DL) approaches. So far, very few efforts have been devoted to exploiting this or other similar computing platforms in on-board remote sensing procedures. This letter explores the use of low-power consumption architectures and DL algorithms for HSI classification. The conducted experimental study reveals that the NVIDIA Jetson Tegra TX2 device offers a good choice in terms of performance, cost, and energy consumption for on-board HSI classification tasks.

This letter introduces a novel remote sensing singleimage superresolution (SR) architecture based on a deep efficient compendium model. The current deep learning-based SR trend stands for using deeper networks to improve the performance. However, this practice often results in the degradation of visual results. To address this issue, the proposed approach harmonizes several different improvements on the network design to achieve state-of-the-art performance when superresolving remote sensing imagery. On the one hand, the proposal combines residual units and skip connections to extract more informative features on both local and global image areas. On the other hand, it makes use of parallelized 1×1 convolutional filters (network in network) to reconstruct the superresolved result while reducing the information loss through the network. Our experiments, conducted using seven different SR methods over the well-known UC Merced remote sensing data set, and two additional GaoFen-2 test images, show that the proposed model is able to provide competitive advantages.

Convolutional neural networks (CNNs) have become a powerful tool for remotely sensed hyperspectral image (HSI) classification due to their great generalization ability and high accuracy.However, owing to the huge amount of parameters that need to be learned and to the complex nature of HSI data itself, these approaches must deal with the important problem of overfitting, which can lead to inadequate generalization and loss of accuracy.In order to mitigate this problem, in this letter we adopt random occlusion, a recently developed data augmentation (DA) method for training CNNs in which the pixels of different rectangular spatial regions in the HSI are randomly occluded, generating training images with various levels of occlusion and reducing the risk of overfitting.Our results with two well-known HSIs reveal that the proposed method helps to achieve better classification accuracy with low computational cost.

The current trend in remote sensing image superresolution (SR) is to use supervised deep learning models to effectively enhance the spatial resolution of airborne and satellite-based optical imagery. Nonetheless, the inherent complexity of these architectures/data often makes these methods very difficult to train. Despite these recent advances, the huge amount of network parameters that must be fine-tuned and the lack of suitable high-resolution remotely sensed imagery in actual operational scenarios still raise some important challenges that may become relevant limitations in the existent earth observation data production environments. To address these problems, we propose a new remote sensing SR approach that integrates a visual attention mechanism within a residual-based network design in order to allow the SR process to focus on those features extracted from land-cover components that require more computations to be superresolved. As a result, the network training process is significantly improved because it aims at learning the most relevant high-frequency information while the proposed architecture allows neglecting the low-frequency features extracted from spatially uninformative earth surface areas by means of several levels of skip connections. Our experimental assessment, conducted using the University of California at Merced and GaoFen-2 remote sensing image collections, three scaling factors, and eight different SR methods, demonstrates that our newly proposed approach exhibits competitive performance in the task of superresolving remotely sensed imagery.

Advances in remote sensing hardware have led to a significantly increased capability for high-quality data acquisition, which allows the collection of remotely sensed images with very high spatial, spectral, and radiometric resolution. This trend calls for the development of new techniques to enhance the way that such unprecedented volumes of data are stored, processed, and analyzed. An important approach to deal with massive volumes of information is data compression, related to how data are compressed before their storage or transmission. For instance, hyperspectral images (HSIs) are characterized by hundreds of spectral bands. In this sense, high-performance computing (HPC) and high-throughput computing (HTC) offer interesting alternatives. Particularly, distributed solutions based on cloud computing can manage and store huge amounts of data in fault-tolerant environments, by interconnecting distributed computing nodes so that no specialized hardware is needed. This strategy greatly reduces the processing costs, making the processing of high volumes of remotely sensed data a natural and even cheap solution. In this paper, we present a new cloud-based technique for spectral analysis and compression of HSIs. Specifically, we develop a cloud implementation of a popular deep neural network for non-linear data compression, known as autoencoder (AE). Apache Spark serves as the backbone of our cloud computing environment by connecting the available processing nodes using a master-slave architecture. Our newly developed approach has been tested using two widely available HSI data sets. Experimental results indicate that cloud computing architectures offer an adequate solution for managing big remotely sensed data sets.

Deep neural networks (DNNs), including convolutional neural networks (CNNs) and residual networks (ResNets) models, are able to learn abstract representations from the input data by considering a deep hierarchy of layers that perform advanced feature extraction. The combination of these models with visual attention techniques can assist with the identification of the most representative parts of the data from a visual standpoint, obtained through more detailed filtering of the features extracted by the operational layers of the network. This is of significant interest for analyzing remotely sensed hyperspectral images (HSIs), characterized by their very high spectral dimensionality. However, few efforts have been conducted in the literature in order to adapt visual attention methods to remotely sensed HSI data analysis. In this paper, we introduce a new visual attention-driven technique for the HSI classification. Specifically, we incorporate attention mechanisms to a ResNet in order to better characterize the spectral-spatial information contained in the data. Our newly proposed method calculates a mask that is applied to the features obtained by the network in order to identify the most desirable ones for classification purposes. Our experiments, conducted using four widely used HSI data sets, reveal that the proposed deep attention model provides competitive advantages in terms of classification accuracy when compared to other state-of-the-art methods.

The classification of hyperspectral images (HSIs) using convolutional neural networks (CNNs) has recently drawn significant attention. However, it is important to address the potential overfitting problems that CNN-based methods suffer when dealing with HSIs. Unlike common natural images, HSIs are essentially three-order tensors which contain two spatial dimensions and one spectral dimension. As a result, exploiting both spatial and spectral information is very important for HSI classification. This paper proposes a new hand-crafted feature extraction method, based on multiscale covariance maps (MCMs), that is specifically aimed at improving the classification of HSIs using CNNs. The proposed method has the following distinctive advantages. First, with the use of covariance maps, the spatial and spectral information of the HSI can be jointly exploited. Each entry in the covariance map stands for the covariance between two different spectral bands within a local spatial window, which can absorb and integrate the two kinds of information (spatial and spectral) in a natural way. Second, by means of our multiscale strategy, each sample can be enhanced with spatial information from different scales, increasing the information conveyed by training samples significantly. To verify the effectiveness of our proposed method, we conduct comprehensive experiments on three widely used hyperspectral data sets, using a classical 2-D CNN (2DCNN) model. Our experimental results demonstrate that the proposed method can indeed increase the robustness of the CNN model. Moreover, the proposed MCMs+2DCNN method exhibits better classification performance than other CNN-based classification strategies and several standard techniques for spectral-spatial classification of HSIs.

Convolutional neural networks (CNNs) have recently exhibited an excellent performance in hyperspectral image classification tasks. However, the straightforward CNN-based network architecture still finds obstacles when effectively exploiting the relationships between hyperspectral imaging (HSI) features in the spectral-spatial domain, which is a key factor to deal with the high level of complexity present in remotely sensed HSI data. Despite the fact that deeper architectures try to mitigate these limitations, they also find challenges with the convergence of the network parameters, which eventually limit the classification performance under highly demanding scenarios. In this paper, we propose a new CNN architecture based on spectral-spatial capsule networks in order to achieve a highly accurate classification of HSIs while significantly reducing the network design complexity. Specifically, based on Hinton's capsule networks, we develop a CNN model extension that redefines the concept of capsule units to become spectral-spatial units specialized in classifying remotely sensed HSI data. The proposed model is composed by several building blocks, called spectral-spatial capsules, which are able to learn HSI spectral-spatial features considering their corresponding spatial positions in the scene, their associated spectral signatures, and also their possible transformations. Our experiments, conducted using five well-known HSI data sets and several state-of-the-art classification methods, reveal that our HSI classification approach based on spectral-spatial capsules is able to provide competitive advantages in terms of both classification accuracy and computational time.

Convolutional neural networks (CNNs) exhibit good performance in image processing tasks, pointing themselves as the current state-of-the-art of deep learning methods. However, the intrinsic complexity of remotely sensed hyperspectral images still limits the performance of many CNN models. The high dimensionality of the HSI data, together with the underlying redundancy and noise, often makes the standard CNN approaches unable to generalize discriminative spectral-spatial features. Moreover, deeper CNN architectures also find challenges when additional layers are added, which hampers the network convergence and produces low classification accuracies. In order to mitigate these issues, this paper presents a new deep CNN architecture specially designed for the HSI data. Our new model pursues to improve the spectral-spatial features uncovered by the convolutional filters of the network. Specifically, the proposed residual-based approach gradually increases the feature map dimension at all convolutional layers, grouped in pyramidal bottleneck residual blocks, in order to involve more locations as the network depth increases while balancing the workload among all units, preserving the time complexity per layer. It can be seen as a pyramid, where the deeper the blocks, the more feature maps can be extracted. Therefore, the diversity of high-level spectral-spatial attributes can be gradually increased across layers to enhance the performance of the proposed network with the HSI data. Our experiments, conducted using four well-known HSI data sets and 10 different classification techniques, reveal that our newly developed HSI pyramidal residual model is able to provide competitive advantages (in terms of both classification accuracy and computational time) over the state-of-the-art HSI classification methods.

The relatively recent availability of operational products from Sentinel-2 and Sentinel-3 missions gives widespread opportunities to combine data collected from different sensors in order to provide products of a higher processing level. Nonetheless, the availability of these products may be affected by multiple factors, such as cloud occlusions, band saturation, geolocation errors or even misaligned detectors. All these anomalies affecting remote sensing data may eventually limit the accessibility to fused products because some of the required information may become partially unavailable for specific areas of interest. In this scenario, the work presented here aims at analyzing the effectiveness of several state-of-the-art regression models in order to restore Sentinel-3 products with partial anomalies from Sentinel-2 integral data. In particular this work investigates three regression methods, two linear-regression method and a non-linear artificial neural networks based method. Obtained results prove that the nonlinear approach and linear RIDGE method are able to carry out a good estimation of S3 from S2 data.

During recent years, many regularization techniques have been proposed to deal with ill-posed problems related to hyperspectral image classification, in which the limited number of training samples contrasts with the very high spectral dimensionality. However, the intrinsic structure of a hyperspectral image often depends on the specific scene and spectrometer, although regularizers like Ridge, LASSO, etc, have been widely used in practical applications. Instead of imposing these regularizers to the probabilistic output of a classifier, this work evaluates the use of extreme learning machines (ELM) with output weights of a single-hidden layer feed-forward neural network (SLFN) regularized with Ridge and LASSO priors, respectively. Experimental results with several real hyperspectral images are conducted to compare the performance and adaptation of these two regularizers with the the original ELM in classification scenarios.

Computational advances have allowed for the development of deep learning (DL) applied to remote sensing data and, particularly, to hyperspectral image (HSI) classification. Deeper architectures are able to establish a better separation of the characteristics of the data, allowing for a better and accurate performance. However, it is known that employing very deep architectures with many abstraction levels can result in a loss of information due to the fact that deep networks often normalize each data individually, without considering the set of adjacent data. To address this issue, this paper implements a self-normalizing neural network (SNN) in order to extract high-level abstract representations without losing information due to the data initialization. The selected activation function (scaled exponential linear units or SELU) normalizes the data considering their neighborhood's information and a special dropout technique (a-dropout), obtaining good classification performance while maintaining the data characteristics across the successive layers. Obtained results show that the proposal improves the performance with few training samples.

Probabilistic topic models have recently shown a great potential in the remote sensing image fusion field, which is particularly helpful in land-cover categorization tasks. This letter first studies the application of probabilistic latent semantic analysis (pLSA) and latent Dirichlet allocation to remote sensing synthetic aperture radar (SAR) and multispectral imaging (MSI) unsupervised land-cover categorization. Then, a novel pLSA-based image fusion approach is presented, which pursues to uncover multimodal feature patterns from SAR and MSI data in order to effectively fuse and categorize Sentinel-1 and Sentinel-2 remotely sensed data. Experiments conducted over two different data sets reveal the advantages of the proposed approach for unsupervised land-cover categorization tasks.

Super-resolution (SR) brings an excellent opportunity to improve a wide range of different remote sensing applications. SR techniques are concerned about increasing the image resolution while providing finer spatial details than those captured by the original acquisition instrument. Therefore, SR techniques are particularly useful to cope with the increasing demand remote sensing imaging applications requiring fine spatial resolution. Even though different machine learning paradigms have been successfully applied in SR, more research is required to improve the SR process without the need of external high-resolution (HR) training examples. This paper proposes a new convolutional generator model to super-resolve low-resolution (LR) remote sensing data from an unsupervised perspective. That is, the proposed generative network is able to initially learn relationships between the LR and HR domains throughout several convolutional, downsampling, batch normalization, and activation layers. Then, the data are symmetrically projected to the target resolution while guaranteeing a reconstruction constraint over the LR input image. An experimental comparison is conducted using 12 different unsupervised SR methods over different test images. Our experiments reveal the potential of the proposed approach to improve the resolution of remote sensing imagery.

Hyperspectral imaging is a widely used technique in remote sensing in which an imaging spectrometer collects hundreds of images (at different wavelength channels) for the same area on the surface of the earth. In the last two decades, several methods (unsupervised, supervised, and semisupervised) have been proposed to deal with the hyperspectral image classification problem. Supervised techniques have been generally more popular, despite the fact that it is difficult to collect labeled samples in real scenarios. In particular, deep neural networks, such as convolutional neural networks (CNNs), have recently shown a great potential to yield high performance in the hyperspectral image classification. However, these techniques require sufficient labeled samples in order to perform properly and generalize well. Obtaining labeled data is expensive and time consuming, and the high dimensionality of hyperspectral data makes it difficult to design classifiers based on limited samples (for instance, CNNs overfit quickly with small training sets). Active learning (AL) can deal with this problem by training the model with a small set of labeled samples that is reinforced by the acquisition of new unlabeled samples. In this paper, we develop a new AL-guided classification model that exploits both the spectral information and the spatial-contextual information in the hyperspectral data. The proposed model makes use of recently developed Bayesian CNNs. Our newly developed technique provides robust classification results when compared with other state-of-the-art techniques for hyperspectral image classification.

Artificial neural networks (ANNs) have been widely used in the analysis of remotely sensed imagery. In particular, convolutional neural networks (CNNs) are gaining more and more attention. Unlike traditional CNNs methods, where the relevant information to classify the elements of a remotely sensed image is extracted only from the last fully-connected layer, the new adaptive deep pyramid matching (ADPM) model [1] takes advantage of the features from all of the convolutional layers. This model allows the optimal fusing weights for different convolutional layers be learned from the data itself. In addition, the combination of CNNs with spatial pyramid pooling (SPP-net) to create the basic deep network allows the use of images with multiple scales, which results in better learning process thanks to the complementary information. The original ADPM method is divided in two parts: the multi-scale deep feature extraction and the ADPM core. In this paper we present a computational improvement of the ADPM core, coding a parallel-multicore version. This strategy is shown to significantly enhance performance in the analysis of remotely sensed data.

The finer spatial, spectral and radiometric resolutions of current and planned sensors are rendering increasingly-high data rates which, coupled with limited on-board storage, downlink bandwidth and receiving ground station availability, make high-throughput, high-performance data-reduction techniques essential in forthcoming missions. On this paper we describe an algorithm well suited to high-dimensional data as those produced by multispectral and hyperspectral sensors, both highly relevant in a broad range of Earth Observation activities with the latter becoming increasingly available and delivering the highest data rates. The performance of parallel implementations of the algorithm on multi-core and GPU architectures is also evaluated.