Deep learning methods have shown promising results in various hyperspectral image (HSI) analysis tasks. Despite these advancements, existing models still struggle to accurately identify fine-classified land cover types on noisy hyperspectral images. Traditional methods have limited performance when extracting features from noisy hyperspectral data. Graph Neural Networks (GNNs) offer an adaptable and robust structure by effectively extracting both spectral and spatial features. However, supervised models still require large quantities of labeled data for effective training, posing a significant challenge. Contrastive learning, which leverages unlabeled data for pre-training, can mitigate this issue by reducing the dependency on extensive manual annotation. To address the issues, we propose WSGraphCL, a weak-strong graph contrastive learning model for HSI classification, and conduct experiments in a few-shot scenario. First, the image is transformed into K-hop subgraphs through a spectral-spatial adjacency matrix construction method. Second, WSGraphCL leverages contrastive learning to pre-train a graph-based encoder on the unlabeled hyperspectral image. We demonstrate that weak-strong augmentations and false negative pairs filtering stabilize pre-training and get good-quality representations. Finally, we test our model with a lightweight classifier on the features with a handful of labels. Experimental results showcase the superior performance of WSGraphCL compared to several baseline models, thereby emphasizing its efficacy in addressing the identified limitations in HSI classification. The code repository will be published on the GitHub project under the URL: https://github.com/zhu-xlab/WSGraphCL.

Mining plays a crucial role in driving industrial development, but unfortunately, it also gives rise to significant environmental pollution. With a keen awareness of this issue, we introduce a small dataset comprising Sentinel-2 RGB images depicting 150 large-scale Chilean mining sites, along with corresponding grayscale masks. This dataset serves to minimize the burdens of annotation and computational resources. The masks are annotated with 8-class labels, facilitating the training of neural networks for semantic segmentation tasks. A model containing a standard U-Net architecture with a ResNet-50 backbone, trained on this dataset performs with a mean intersection over union (meanIoU) score of 58 per cent. To improve the performance of our model, we employ pre-trained backbones. Utilizing a ResNet-50 backbone pre-trained on the 2012 ILSVRC ImageNet dataset elevates the meanIoU score to an impressive 71.7 per cent. Encouraged by such substantial improvements from a pre-trained backbone trained on natural images, we now propose the adoption of a ResNet-50 backbone, pre-trained through a self-supervised learning (SSL) method called MoCo, leveraging the SSL4EO-S12 dataset. This dataset contains extensive multimodal multitemporal unlabeled patch triplets, integrating Dual-pol SAR, S2-L1C TOA multispectral, and S2-L2A BOA multispectral data. Employing a ResNet-50 backbone solely pre-trained on one modality, S2-L2A RGB images from the SSL4EO-S12 dataset yield a striking meanIoU score of 77.5 per cent on our curated Chilean mining sites' dataset. It is worth noting that the remarkable performance boost achieved through SSL in our experimental setup underscores the significant value of SSL pre-training using specific remote sensing data/images, particularly in remote sensing segmentation tasks. This approach substantiates our initial objective of mitigating the adverse environmental impacts of mining operations.

Hyperspectral image (HSI) classification is gaining a lot of momentum in present time because of high inherent spectral information within the images. However, these images suffer from the problem of curse of dimensionality and usually require a large number samples for tasks such as classification, especially in supervised setting. Recently, to effectively train the deep learning models with minimal labelled samples, the unlabeled samples are also being leveraged in self-supervised and semi-supervised setting. In this work, we leverage the idea of semi-supervised learning to assist the discriminative self-supervised pretraining of the models. The proposed method takes different augmented views of the unlabeled samples as input and assigns them the same pseudo-label corresponding to the labelled sample from the downstream task. We train our model on two HSI datasets, anemly Houston dataset (from data fusion contest, 2013) and Pavia university dataset, and show that the proposed approach performs better than self-supervised approach and supervised training.

The Landsat program is the longest-running Earth observation program in history, with 50+ years of data acquisition by 8 satellites. The multispectral imagery captured by sensors onboard these satellites is critical for a wide range of scientific fields. Despite the increasing popularity of deep learning and remote sensing, the majority of researchers still use decision trees and random forests for Landsat image analysis due to the prevalence of small labeled datasets and lack of foundation models. In this paper, we introduce SSL4EO-L, the first ever dataset designed for Self-Supervised Learning for Earth Observation for the Landsat family of satellites (including 3 sensors and 2 product levels) and the largest Landsat dataset in history (5M image patches). Additionally, we modernize and re-release the L7 Irish and L8 Biome cloud detection datasets, and introduce the first ML benchmark datasets for Landsats 4–5 TM and Landsat 7 ETM+ SR. Finally, we pre-train the first foundation models for Landsat imagery using SSL4EO-L and evaluate their performance on multiple semantic segmentation tasks. All datasets and model weights are available via the TorchGeo library, making reproducibility and experimentation easy, and enabling scientific advancements in the burgeoning field of remote sensing for a multitude of downstream applications.

Deep learning has proven to be a very effective approach for Hyperspectral Image (HSI) classification. However, deep neural networks require large annotated datasets to generalize well. This limits the applicability of deep learning for HSI classification, where manually labelling thousands of pixels for every scene is impractical. In this paper, we propose to leverage Self Supervised Learning (SSL) for HSI classification. We show that by pre-training an encoder on unlabeled pixels using Barlow-Twins, a state-of-the-art SSL algorithm, we can obtain accurate models with a handful of labels. Experimental results demonstrate that this approach significantly outperforms vanilla supervised learning.

Homomorphic Encryption is one of the most promising techniques to deal with privacy concerns, which is raised by remote deep learning paradigm, and maintain high classification accuracy. However, homomorphic encryption-based solutions are characterized by high overhead in terms of both computation and communication, which limits their adoption in pervasive health monitoring applications with constrained client-side devices. In this paper, we propose PReDIHERO, an improved privacy-preserving solution for remote deep learning inferences based on homomorphic encryption. The proposed solution applies a reversible obfuscation technique that successfully protects sensitive information, and enhances the client-side overhead compared to the conventional homomorphic encryption approach. The solution tackles three main heavyweight client-side tasks, namely, encryption and transmission of private data, refreshing encrypted data, and outsourcing computation of activation functions. The efficiency of the client-side is evaluated on a healthcare dataset and compared to a conventional homomorphic encryption approach. The evaluation results show that PReDIHERO requires increasingly less time and storage in comparison to conventional solutions when inferences are requested. At two hundreds inferences, the improvement ratio could reach more than 30 times in terms of computation overhead, and more than 8 times in terms of communication overhead. The same behavior is observed in sequential data and batch inferences, as we record an improvement ratio of more than 100 times in terms of computation overhead, and more than 20 times in terms of communication overhead.