Abstract Detecting airborne dust in standard RGB images presents significant challenges. Nevertheless, the monitoring of airborne dust holds substantial potential benefits for climate protection, environmentally sustainable construction, scientific research, and various other fields. To develop an efficient and robust algorithm for airborne dust monitoring, several hurdles have to be addressed. Airborne dust can be opaque or translucent, exhibit considerable variation in density, and possess indistinct boundaries. Moreover, distinguishing dust from other atmospheric phenomena, such as fog or clouds, can be particularly challenging. To meet the demand for a high-performing and reliable method for monitoring airborne dust, we introduce DustNet++, a neural network designed for dust density estimation. DustNet++ leverages feature maps from multiple resolution scales and semantic levels through window and grid attention mechanisms to maintain a sparse, globally effective receptive field with linear complexity. To validate our approach, we benchmark the performance of DustNet++ against existing methods from the domains of crowd counting and monocular depth estimation using the Meteodata airborne dust dataset and the URDE binary dust segmentation dataset. Our findings demonstrate that DustNet++ surpasses comparative methodologies in terms of regression and localization capabilities.

This study compares two machine learning pixel classifiers with classical approaches for detecting camouflage targets in hyperspectral data. Recent applications of machine learning for hyperspectral data exploitation show good land cover classification results. However, the spectral differences between the classes in those studies are usually very high. We evaluate their performance for classifying targets with similar spectra, specifically camouflage objects. The machine learning results are compared to the established ACE and SVM multiclass classifiers. Input parameters for all approaches, such as training data, spectral class references, and background information, are extracted from the same label set in a single flight line. The evaluation is carried out on 15 different datasets of the same area. We evaluate the results on hyperspectral data from an elaborate measurement campaign using a drone-borne HySpex Mjolnir VS-620 using the combined VNIR and SWIR information. The results show that the SVM produces the best overall accuracy in this experiment with highly unbalanced classes. The machine learning approaches PGBS-HSI and SpectralFormer show better results for the classes with fewer samples. The ACE has the best average but lowest overall accuracy among the tested methods. The findings of this study contribute to understanding the strengths and limitations of machine learning and classical approaches for camouflage target detection in hyperspectral data.

This work addresses the challenge of including the spatial dimension into the autoencoder models for lossy compression of different spatially independent and unknown hyperspectral datasets acquired by space-borne hyperspectral sensors. We propose two different 3D-Hybrid Convolutional Autoencoder models with increased compression rates compared to 1D methods that can compress and reconstruct hyperspectral data with arbitrary spectral dimensionality. The architecture of the first 3D-Hybrid model consists of the A1D-CAE in combination with the 2D-CAE. The second 3D-Hybrid model includes the adaptive 1D-CAE and a 3D-CAE. The evaluation of the reconstruction accuracy is measured by comparing the spectral angle and the peak signal-to-noise ratio between the original and the reconstructed data and structural similarity index measure. We show the high transferability and generalizability of our 3D-Hybrid models on different PRISMA datasets. The 3D-Hybrid model is compared with the SSCNet2Dbased on a 2D-CAE and a 3D-CAE model. The findings of this study contribute to understanding the strengths and limitations of machine learning-based compression methods for jointly compressing spectral and spatial information.

The task of Land Cover Classification (LCC) is a central activity because it serves as an instrument for decision-making processes. During the last few years, efforts were made to fuse Hyperspectral (HS) and Light Detection and Ranging (LiDAR) data for creating proficient classifiers. This fusion enables high-resolution classifications on scenes with spectrally similar categories. Additionally, its resolution mainly involves the combination of image-level features acquired during the fusion and subsequent classification. However, another perspective to solve this challenge is using self-supervised features to learn single-modal classifiers and then fusing their individual decisions. The current method approaches the alternative above by resolving several self-supervised tasks, using their weights in individual classifiers, and then combining their outcomes to solve the LCC. It starts by training a Siamese network that applies implicit contrastive learning on augmented views of HS data to learn the semantics of similarities. Then, the method learns two Denoising Autoencoders (DAEs) separately to remove noise from artificially corrupted HS and LiDAR patches. The denoising allows the networks to extract the most relevant image-level features. Subsequently, three instances of a ResNet50-based classifier employ the individually gained self-supervised features to train with a fraction of the available labels. The Decision Fusion Module (DFM) takes each learned classifier’s weights and fuses their individual decisions to compute the final classification. The validation employs two benchmark datasets. Experiments show that the learned self-supervised representations support the method to achieve proficient classification results.

This work discusses three individual camouflage experiments from a drone-based hyperspectral measurement campaign conducted in 2021. The experiments were designed to provide insight into different scenarios of camouflage classification and detection of camouflaged objects. The first experiment demonstrates an approach to detect different objects under camouflage using spectral unmixing. The second experiment presents the performance of commonly used hyperspectral classifiers for camouflage detection with respect to natural illumination changes throughout the day. Finally, the third experiment evaluates the effect of moisture on camouflage detection. For all experiments, we discuss the conditions under which hyperspectral data together with established detection and classification approaches can be used to robustly locate camouflage nets, and when detection is impaired.

This work addresses the challenge of transferability of autoencoder models for lossy compression of different spatially independent and unknown hyperspectral datasets acquired from different space sensor platforms. We propose an adaptive 1D convolutional autoencoder architecture that can compress and recover spectral signatures with different numbers of bands. We demonstrate the transferability of the 1D CAE to different sensors by applying different unknown hyperspectral datasets acquired by different sensor platforms. The evaluation of the reconstruction accuracy is measured by comparing the spectral angle and the signal-to-noise ratio between the original and the reconstructed data. We show the high transferability and generalizability of our A1D-CAE model for compression rates cR= 4 on different datasets from the satellite-based PRISMA, DESIS, EnMap and HYPSO-1 sensors. The results show that the proposed A1D-CAE architecture is capable of processing hyperspectral data from multiple sensor sources with different characteristics while achieving high reconstruction accuracy.

Airborne dust has a broad impact from climate to human health. Extensive dust monitoring can lead to identifying environmental hazards and developing mitigation strategies. However, conventional dust measuring devices are usually expensive and limited for the monitoring of the spatial characteristics of dust. Available RGB camera systems might be a potential tool for the measurement of these spatial characteristics, but the automatic detection of airborne dust within these images is not well-researched. The challenges for the required algorithm for such an automatic detection are manifold, including the opaqueness, the wide range of possible density levels, the visual similarity to effects like smoke or clouds, and the fuzzy boundaries of airborne dust. In order to face these challenges in the underexplored research field of detecting airborne dust in terrestrial RGB images, we propose DeepDust. DeepDust is a dust density estimation neural network and exploits convolutional-based multi-level embeddings to merge features from different resolutions and semantic levels. Due to the absence of existing methods in our research field, we compare results achieved by our DeepDust with techniques from the crowd counting and monocular depth estimation domain on the Meteodata dust dataset. Our DeepDust outperforms the other evaluated approaches regarding regression ability by a wide margin.

HyperThun’22 was a multi-sensor and multi-temporal camouflage detection campaign with drone-carried hyper-spectral, thermal, and RGB instruments. In more than 20 flights, various military targets were imaged with the purpose of analysing detection rates, camouflage transparency, and system performances. This article presents the campaign design, the data processing, and first data insights. Preliminary results show the potential of the acquired data for promising studies.

The growing availability of hyperspectral remote sensing data, specifically from the new hyperspectral satellite missions, requires efficient data compression due to limitations in bandwidth and available storage space while simultaneously preserving the spectral characteristics. Machine learning approaches are a powerful way to address this challenge, but they are usually tailored to only work on one specific sensor. This work addresses the challenge of transferability of autoencoder models for lossy compression of spatially independent and unknown hyperspectral datasets acquired from different sensor platforms. We propose the CompNext1D, an advanced multi-stage adaptive network based on the architecture of the A1D-CAE. The characteristic of the A1D-CAE allows pre-training on a large dataset with wide spectral variability and transferability to other sensor data, e.g., with potentially limited data availability. The compression performance of the CompNext1D is enhanced by using image statistics and shows a high degree of transferability to unknown spectral signatures. We evaluate the reconstruction accuracy with three experiments of increasing complexity. The evaluation is based on the reconstruction accuracy using the SA, SNR, PSNR, and SSIM metrics, and the results are compared to other learning-based lossy compression techniques.We demonstrate the high transferability and generalizability of our A1D-CAE and CompNext1D for compression rates fromcR= 4 tocR≈ 100 on hyperspectral data from different sensors and carrier platforms. The CompNext1D architecture performs well in compressing hyperspectral data from multiple sensor sources with different characteristics while achieving higher reconstruction accuracy compared to state-of-the-art methods.

The task of semantic segmentation plays a central role in the analysis of remotely sensed imagery. This relevance is reflected in the act of classifying each image pixel belonging to a particular class. This allows the acquisition of semantic knowledge in form of a classification map, which facilitates decision-making processes. Nowadays, the task of semantic segmentation is mainly solved with Supervised pre-training. It needs plenty of labels to learn a mapping function, which produces useful features. As alternative, Self-supervised learning (SSL) techniques entirely explore the data, find supervision signals and solve a challenge called Pretext task for coming upon robust representations. The current work investigates Image Colorization (IC) as Pretext task to learn feature representations, which will be transferred to an U-Net for predicting semantic segmentations of urban scenes. The study examines two benchmark datasets for validation and generation of classification maps. The results show that the learned features through colorization achieve accurate segmentation results. This was possible both using unlabeled ImageNet training data and the actual datasets. These contain up to half a million examples, which represents a modest amount compared to the number of annotated images present in ImageNet.

Nonlinear effects in hyperspectral (HS) remote sensing data, caused by shadows, varying illumination conditions, as well as by directional reflectance variations, may lead to inaccurate land cover classification. Including additional features of a simultaneously collected digital elevation model (DEM) generally improves the results. In this paper, we apply the Nonlinear Feature Normalization (NFN) to a weighted concatenation of HS channels and different sets of features derived from DEMs to improve the classification accuracy. The evaluation is performed on two data sets, where, the labeled data for one of them was derived using an interactive approach based on unsupervised classification. Using sensor data fusion and NFN transformation improved classification accuracy from a Cohens κ of 0.6 to values over 0.8.

In the last decades, hyperspectral imaging (HSI) has become an appealing field of remote sensing because of its richness in information. However, HSI suffers from redundancy and high computational complexity. To address this challenge, we present Policy Gradient Band Search - an intuitive and straightforward search strategy for dimensionality reduction of hyperspectral data. The family of policy gradient algorithms is a widely used and successful model-free approach in the reinforcement learning framework and is here applied for HSI band selection. In order to exhibit the effectiveness of our method, we evaluate our approach under consideration the spatial-spectral relationship in combination with a 3D-CNN based HSI classification network. Our approach yields satisfying empirical results in three HSI datasets, which are Pavia University, Salinas and Greding. The overall accuracy of 98.14%,96.64%,99.84% was separately accomplished on these datasets while being restricted by utilizing only four chosen bands.

Semantic segmentation is an important computer vision task for the analysis of aerial imagery in many remote sensing applications. Due to the large availability of data it is possible to design efficient convolutional neural network based deep learning models for this purpose. But these methods usually show a weak performance when they are applied without any modifications to data from another domain with different characteristics relating to aspects concerning the sensor or environmental influences. To improve the performance of these methods domain adaptation approaches can be employed. In the following work, we want to present a method for unsupervised domain adaptation for semantic segmentation. We trained an encoder-decoder model on the source domain dataset as task application and adjusted the network to the target domain. The adaptation process is based on a style transfer component, which is realized using a cycle-consistent adversarial network. Through a continuous adaptation of the task model we achieved a higher generalization of the network and increased the task method performance on the target domain.

Hyperspectral target detection experiments under nonideal conditions are scarce. An extensive multi-scale and multi-temporal field experiment was designed towards the goal of knowledge expansion under such circumstances. A range of camouflage materials and specific targets of interest were placed in a realistic natural environment with vegetation cover and varying illumination. In several experiments, aspects like changes in the sun position, variable moisture, and relocations of targets were analysed. Using an aircraft-based and a drone-based imaging spectrometer, the target scenarios were mapped at different daytimes. The data were radiometrically, atmospherically and geometrically processed to allow subsequent data analysis. First insights deliver promising results.

Semantic segmentation is an important analysis task for the investigation of aerial imagery. Recently, the arise of convolutional neural networks has increased the performance of computer vision methods considerably. But the success of deep learning applications mostly relies on the availability of sufficiently large training datasets. However, the manual annotation of images is time consuming and needs human effort. To reduce the necessary amount of training data it is possible to fine-tune a model which is pre-trained on a different larger dataset. But usually orthophotos are affected by weather and sensor dependent light conditions. Additionally, such images are composed of imbalanced classes which leads to poor pixel-wise classification results for sparsely represented labels. In this paper we propose a convolutional neural network based domain adaptation method for semantic segmentation. The encoder-decoder structure uses adaptation modules and an alternately training procedure to adapt the network to the target domain. We employ the large ISPRS Potsdam dataset as source domain to train a base model and adapt it using very few samples. We compared our method to the common fine-tuning approach and evaluated the results for a decreasing number of training samples. We observed an improvement of the average overall prediction accuracy but especially for the sparsely represented vehicle class.

Domain adaptation in remote sensing aims at the automatic knowledge transfer between a set of multitemporal and multisource images. This process is often impaired by nonlinear effects in the data, e.g., varying illumination conditions, different viewing angles, and geometry-dependent reflection. In this paper, we introduce the Nonlinear Feature Normalization (NFN), a fast and robust way to align the spectral characteristics of multiple hyperspectral data sets. NFN employs labeled training spectra for the different classes in an image to describe the corresponding underlying low-dimensional manifold structure. A linear basis for data representation is defined by arbitrary class reference vectors, and the image is aligned to the new basis in the same space. This results in samples of the same class being pulled closer together and samples of different classes pushed apart. NFN transforms the data in its original domain, preserving physical interpretability. We use the continuous invertibility of NFN to derive the NFN Alignment (NFNalign) transformation, which can be used for domain adaptation, by transforming one data set to the domain of a chosen reference. The evaluation is performed on multiple hyperspectral data sets as well as our new benchmark for multitemporal hyperspectral data. In a first step, we show that the NFN transformation successfully mitigates nonlinear effects by comparing classification of the linear Spectral Angle Mapper on original and transformed data. Finally, we demonstrate successful domain adaptation with NFNalign by applying it to the task of hyperspectral data preprocessing. The evaluation shows that our approach for alignment of multitemporal data produces high-spectral similarity and successfully allows knowledge transfer, e.g., of classifier models and training data.

At Fraunhofer IOSB, a concept for a near real-time airborne 3D mapping system for disaster management and security applications was proposed [1]. In addition to an Airborne Laser Scanner (ALS), a RGB camera facing nadir is installed on that platform. Subsequent RBG pictures are used for reconstructing urban scenes, and an ALS point cloud is used as a reference data set to evaluate this work. For in-flight applications, the implemented 3D reconstruction from overlapping nadir images should operate in near real-time. This paper focuses on computational performance and quality considerations of 3D reconstruction of urban scenes with aerial images. Different subpixel disparity levels are considered for generating 3D models. The run-time and quality for depth map generation with two methods are evaluated as well. The PatchMatch Stereo (PMS) algorithm is compared with the Semi-Global Matching (SGM), the most popular algorithm for 3D reconstruction.

Non-linear effects in hyperspectral data are the result of varying illumination conditions, angular dependencies of reflection, shadows and multiple scattering of incident light. Common classification algorithms like Spectral Angular Mapper (SAM) and Adaptive Coherence Estimator (ACE) struggle to produce good results under these conditions. In this paper, we evaluate our fast Semi-supervised Invertible Manifold Alignment, introduced in [1], on multiple commonly available hyperspectral remote sensing data sets. Additionally, we test it on our new benchmark data set for multitemporal analysis. We show that linear SAM classification on SIMA-transformed data is superior to linear classification on the original data in all cases. Also, SIMA-transformation with subsequent SAM classification produces comparable results to a multi-class Support Vector Machine (SVM), with the benefit of maintaining physical interpretability of the transformed data.

This paper suggests using color-guided depth enhancement algorithms of computer vision to improve the resolution of airborne laser scanning (ALS) point clouds for remote sensing applications. We use co-registered high resolution color images with nadir view to enhance the ALS data; and perform quantitative evaluation in form of RMSE considering the whole depth image as well as the depth discontinuities only. Investigated methods include joint bilateral filtering, Markov Random Field (MRF) optimization with first and second order smoothness terms, and anisotropic diffusion. RMSE results on discontinuities indicate that detail improvement performance of the selected methods on the depth discontinuities is not on a satisfactory level for airborne data. Anisotropic diffusion and MRF optimization are promising to provide better results with further adjustments on the smoothness terms.

Different methods for the detection for hydrocarbons in aerial hyperspectral images are analyzed in this study. The scope is to find a practical method for airborne oil spill mapping on land. Examined are Hydrocarbon index and Hydrocarbon detection index. As well as spectral reidentification algorithms, like Spectral angle mapper, in comparison to the indices. The influence of different ground coverage and different hydrocarbons was tested and evaluated. A ground measurement campaign was conducted with controlled contaminations and manual definition of ground truth data, to evaluate the performance of the detection methods. Additionally, the discriminability between wet ground and oil-contaminated ground is investigated, along with the temporal influence on oil spill detection.

Nonlinear effects in hyperspectral data complicate classification and other data analysis procedures. Transforming the data onto manifolds can help to improve the results while simultaneously reducing the dimensionality due to the high correlation among the spectral bands. Methods like ISOMAP or Locally Linear Embedding are not ideal when the data is degraded by noise. In this paper, a method is introduced to automatically generate support points for skeletonizing a high-dimensional point cloud. The skeleton is identified with multiple signal source variations of distinct materials and can be used to transform the data to improve further analysis procedures.

This paper suggests a method for automatic in-flight boresight calibration of pushbroom scanner images, using an on-line system with broadband data downlink and near realtime georeferencing of the pushbroom image data. Georeferencing accuracy may decrease during long image acquisition flights due to instable atmospheric conditions, which may lead to geometric changes in the flight platform. Orthorectification of (hyperspectral) pushbroom scanner data demands the knowledge of the extrinsic orientation parameters for every exposure. The most crucial parameters for the transformation of the pose obtained by the inertial navigation system (INS) into the projection center of the imaging sensor are the boresight angles. Utilizing a performant ray tracing algorithm and a digital elevation model (DEM), these parameters can be estimated even while flying in uneven and uninhabited areas. Tie points for solving an extended collinear equation are extracted automatically by the SURF algorithm.

In the last few years, unmixing of hyperspectral data has become of major importance. The high spectral resolution results in a loss of spatial resolution. Thus, spectra of edges and small objects are composed of mixtures of their neighboring materials. Due to the fact that supervised unmixing is impossible for extensive data sets, the unsupervised Nonnegative Matrix Factorization (NMF) is used to automatically determine the pure materials, so called endmembers, and their abundances per sample [1]. As the underlying optimization problem is nonlinear, a good initialization improves the outcome [2]. In this paper, several methods are combined to create an algorithm for fully unsupervised spectral unmixing. Major part of this paper is an initialization method, which iteratively calculates the best possible candidates for endmembers among the measured data. A termination condition is applied to prevent violations of the linear mixture model. The actual unmixing is performed by the multiplicative update from [3]. Using the proposed algorithm it is possible to perform unmixing without a priori studies and accomplish a sparse and easily interpretable solution. The algorithm was tested on different hyperspectral data sets of the sensor types AISA Hawk and AISA Eagle.