Masked autoencoders employ random masking to effectively reconstruct input images using self-supervised techniques, which allows for efficient training on large datasets. However, the random masking strategy does not adequately tap into information encapsulated within high-dimensional hyperspectral satellite imagery that is used in several domains. We propose a novel masking strategy, HOGMAE, based on the Histogram of Oriented Gradients that incorporates rich information inherent within satellite images during the mask creation step. Our experiments, over a hyperspectral satellite dataset, demonstrate the effectiveness of our methodology.

The proliferation of foundation models, pretrained on large-scale unlabeled datasets, has emerged as an effective approach in creating adaptable and reusable architectures that can be leveraged for various downstream tasks using satellite observations. However, their direct application to hyperspectral remote sensing remains challenging due to inherent spectral disparities and the scarcity of available observations. In this work, we present HyperKD, a novel knowledge distillation framework that enables transferring learned representations from a teacher model into a student model for effective development of a foundation model on hyperspectral images. Unlike typical knowledge distillation frameworks, which use a complex teacher to guide a simpler student, HyperKD enables an inverse form of knowledge transfer across different types of spectral data, guided by a simpler teacher model. Building upon a Masked Autoencoder (MAE) with a Vision Transformer (ViT) backbone, HyperKD distills knowledge from Prithvi (a ViT-based MAE geospatial foundation model trained on lower-dimensional multispectral data) into a student tailored for EnMAP hyperspectral imagery. HyperKD addresses the inverse domain adaptation problem with spectral gaps by introducing a feature-based strategy that includes spectral range-based channel alignment, spatial featureguided masking, and an enhanced loss function tailored for hyperspectral images. HyperKD bridges the substantial spectral domain gap, enabling the effective use of pretrained foundation models for geospatial applications. Extensive experiments show that HyperKD significantly improves representation learning in MAEs, leading to enhanced reconstruction fidelity and more robust performance on downstream tasks such as land cover classification, crop type identification, and soil organic carbon prediction, underpinning the potential of knowledge distillation frameworks in remote sensing analytics with hyperspectral imagery.

Masked Autoencoders struggle with hyperspectral satellite imagery containing 200+ spectral bands, as uniform masking across all channels obscures critical spatial-spectral relationships. We introduce TerraMAE, which employs an adaptive channel grouping strategy to organize bands into statistically coherent groups with independent masking. Together with a customized loss function, this data-driven grouping strategy enables TerraMAE to learn robust spatial-spectral representations from unlabeled HSI. Experiments demonstrate that TerraMAE significantly outperforms baseline Masked Autoencoder and supervised ResNet-50 on soil texture prediction, achieving 15.7% and 6.6% lower error, respectively.

Human action recognition has been explored in healthcare, sports, and entertainment, with a recent shift toward manufacturing settings for monitoring assembly tasks. Identifying assembly actions is crucial for improving human-robot collaboration and optimizing the assembly process. However, the complexity of assembly tasks poses challenges for action recognition methods, with single-modality methods struggling to capture the complex dynamics and context. We proposed the multimodal ConvLSTM-AssNet and C3D-AssNet methods, which use RGB, RGB-A, and depth data. The models are tested in single, double, and triple stream configurations, with attention mechanisms integrated to focus on relevant features. The proposed models are evaluated on the HA4M dataset. Attention-Guided C3D-AssNet is most accurate for single (RGB-A: 97.10%) and double streams (RGB-A + Depth: 98.84%), while ConvLSTM-AssNet performs best for triple streams (RGB + RGB-A + Depth: 97.30%). This research advances multimodal assembly action recognition for manufacturing applications.

Accurate estimation of soil moisture is crucial for efficient agricultural management and environmental monitoring. However, the task of predicting soil moisture levels becomes challenging in regions with limited data availability. In this study, we propose a knowledge distillation-based deep learning approach to enhance soil moisture prediction with machine learning apporach using the low resolution but wide coverage soil moisture Active Passive (SMAP) satellite data.Our framework leverages the knowledge distillation, where a high-capacity teacehr model (VGG13) which is pre-traineed on a large dataset (SMAP) and a lightweight student model (ResNet8) which is then trained on sensor-based highly accurate but extremely sparse station data. The student model benefits from the distilled knowledge of the teacher model, acquiring a deeper understanding of the underlying patterns and relationships in the data.The space-efficient student model significantly reduces the inference time with high prediction accuracy and demonstrates the potential benefit to agricultural management, water resource planning, and ecological studies by providing accurate and reliable soil moisture predictions in data-scarce regions. Our findings reveal how to identify performant settings for achieving the best trade-off between accuracy and model complexity.

Multi-spectral satellite images that remotely sense the Earth's surface at regular intervals are often contaminated due to occlusion by clouds. Remote sensing imagery captured via satellites, drones, and aircraft has successfully influenced a wide range of fields such as monitoring vegetation health, tracking droughts, and weather forecasting, among others. Researchers studying the Earth's surface are often hindered while gathering reliable observations due to contaminated reflectance values that are sensitive to thin, thick, and cirrus clouds, as well as their shadows. In this study, we propose a deep learning network architecture, CloudNet, to alleviate cloud-occluded remote sensing imagery captured by Landsat-8 satellite for both visible and non-visible spectral bands. We propose a deep neural network model trained on a distributed storage cluster that leverages historical trends within Landsat-8 imagery while complementing this analysis with high-resolution Sentinel-2 imagery. Our empirical benchmarks profile the efficiency of the CloudNet model with a range of cloud-occluded pixels in the input image. We further compare our CloudNet's performance with state-of-the-art deep learning approaches such as SpAGAN and Resnet. We propose a novel method, dynamic hierarchical transfer learning, to reduce computational resource requirements while training the model to achieve the desired accuracy. Our model regenerates features of cloudy images with a high PSNR accuracy of 34.28 dB.

Invasive ductal carcinoma (IDC) is the most common type of breast cancer. Every year a numerous number of women in this world are diagnosed as having IDC. Accurately detecting IDC is a time consuming and challenging task as the pathologists need to focus on the specific regions of whole slide images (WSI) that contain IDC. Precise and early diagnosis of IDC is a must because it helps to estimate the subsequent tumor aggressiveness that can be caused by this type of breast cancer. The goal of this research is to create an automated system that will analyze the whole mount slide images of breast cancer specimens to indicate the exact positions of IDC inside of the slides and give a decision based on the results. A multilayered convolutional neural network is designed which is trained over a large number of whole slide images. The dataset consists of 162 cases of patients diagnosed with IDC. We found an accuracy of 89.34% in f1 score using convolutional neural network to achieve the state of the art result on IDC classification.