This study compares two machine learning pixel classifiers with classical approaches for detecting camouflage targets in hyperspectral data. Recent applications of machine learning for hyperspectral data exploitation show good land cover classification results. However, the spectral differences between the classes in those studies are usually very high. We evaluate their performance for classifying targets with similar spectra, specifically camouflage objects. The machine learning results are compared to the established ACE and SVM multiclass classifiers. Input parameters for all approaches, such as training data, spectral class references, and background information, are extracted from the same label set in a single flight line. The evaluation is carried out on 15 different datasets of the same area. We evaluate the results on hyperspectral data from an elaborate measurement campaign using a drone-borne HySpex Mjolnir VS-620 using the combined VNIR and SWIR information. The results show that the SVM produces the best overall accuracy in this experiment with highly unbalanced classes. The machine learning approaches PGBS-HSI and SpectralFormer show better results for the classes with fewer samples. The ACE has the best average but lowest overall accuracy among the tested methods. The findings of this study contribute to understanding the strengths and limitations of machine learning and classical approaches for camouflage target detection in hyperspectral data.

This work addresses the challenge of including the spatial dimension into the autoencoder models for lossy compression of different spatially independent and unknown hyperspectral datasets acquired by space-borne hyperspectral sensors. We propose two different 3D-Hybrid Convolutional Autoencoder models with increased compression rates compared to 1D methods that can compress and reconstruct hyperspectral data with arbitrary spectral dimensionality. The architecture of the first 3D-Hybrid model consists of the A1D-CAE in combination with the 2D-CAE. The second 3D-Hybrid model includes the adaptive 1D-CAE and a 3D-CAE. The evaluation of the reconstruction accuracy is measured by comparing the spectral angle and the peak signal-to-noise ratio between the original and the reconstructed data and structural similarity index measure. We show the high transferability and generalizability of our 3D-Hybrid models on different PRISMA datasets. The 3D-Hybrid model is compared with the SSCNet2Dbased on a 2D-CAE and a 3D-CAE model. The findings of this study contribute to understanding the strengths and limitations of machine learning-based compression methods for jointly compressing spectral and spatial information.

The task of Land Cover Classification (LCC) is a central activity because it serves as an instrument for decision-making processes. During the last few years, efforts were made to fuse Hyperspectral (HS) and Light Detection and Ranging (LiDAR) data for creating proficient classifiers. This fusion enables high-resolution classifications on scenes with spectrally similar categories. Additionally, its resolution mainly involves the combination of image-level features acquired during the fusion and subsequent classification. However, another perspective to solve this challenge is using self-supervised features to learn single-modal classifiers and then fusing their individual decisions. The current method approaches the alternative above by resolving several self-supervised tasks, using their weights in individual classifiers, and then combining their outcomes to solve the LCC. It starts by training a Siamese network that applies implicit contrastive learning on augmented views of HS data to learn the semantics of similarities. Then, the method learns two Denoising Autoencoders (DAEs) separately to remove noise from artificially corrupted HS and LiDAR patches. The denoising allows the networks to extract the most relevant image-level features. Subsequently, three instances of a ResNet50-based classifier employ the individually gained self-supervised features to train with a fraction of the available labels. The Decision Fusion Module (DFM) takes each learned classifier’s weights and fuses their individual decisions to compute the final classification. The validation employs two benchmark datasets. Experiments show that the learned self-supervised representations support the method to achieve proficient classification results.

This work discusses three individual camouflage experiments from a drone-based hyperspectral measurement campaign conducted in 2021. The experiments were designed to provide insight into different scenarios of camouflage classification and detection of camouflaged objects. The first experiment demonstrates an approach to detect different objects under camouflage using spectral unmixing. The second experiment presents the performance of commonly used hyperspectral classifiers for camouflage detection with respect to natural illumination changes throughout the day. Finally, the third experiment evaluates the effect of moisture on camouflage detection. For all experiments, we discuss the conditions under which hyperspectral data together with established detection and classification approaches can be used to robustly locate camouflage nets, and when detection is impaired.

This work addresses the challenge of transferability of autoencoder models for lossy compression of different spatially independent and unknown hyperspectral datasets acquired from different space sensor platforms. We propose an adaptive 1D convolutional autoencoder architecture that can compress and recover spectral signatures with different numbers of bands. We demonstrate the transferability of the 1D CAE to different sensors by applying different unknown hyperspectral datasets acquired by different sensor platforms. The evaluation of the reconstruction accuracy is measured by comparing the spectral angle and the signal-to-noise ratio between the original and the reconstructed data. We show the high transferability and generalizability of our A1D-CAE model for compression rates cR= 4 on different datasets from the satellite-based PRISMA, DESIS, EnMap and HYPSO-1 sensors. The results show that the proposed A1D-CAE architecture is capable of processing hyperspectral data from multiple sensor sources with different characteristics while achieving high reconstruction accuracy.

HyperThun’22 was a multi-sensor and multi-temporal camouflage detection campaign with drone-carried hyper-spectral, thermal, and RGB instruments. In more than 20 flights, various military targets were imaged with the purpose of analysing detection rates, camouflage transparency, and system performances. This article presents the campaign design, the data processing, and first data insights. Preliminary results show the potential of the acquired data for promising studies.

The growing availability of hyperspectral remote sensing data, specifically from the new hyperspectral satellite missions, requires efficient data compression due to limitations in bandwidth and available storage space while simultaneously preserving the spectral characteristics. Machine learning approaches are a powerful way to address this challenge, but they are usually tailored to only work on one specific sensor. This work addresses the challenge of transferability of autoencoder models for lossy compression of spatially independent and unknown hyperspectral datasets acquired from different sensor platforms. We propose the CompNext1D, an advanced multi-stage adaptive network based on the architecture of the A1D-CAE. The characteristic of the A1D-CAE allows pre-training on a large dataset with wide spectral variability and transferability to other sensor data, e.g., with potentially limited data availability. The compression performance of the CompNext1D is enhanced by using image statistics and shows a high degree of transferability to unknown spectral signatures. We evaluate the reconstruction accuracy with three experiments of increasing complexity. The evaluation is based on the reconstruction accuracy using the SA, SNR, PSNR, and SSIM metrics, and the results are compared to other learning-based lossy compression techniques.We demonstrate the high transferability and generalizability of our A1D-CAE and CompNext1D for compression rates fromcR= 4 tocR≈ 100 on hyperspectral data from different sensors and carrier platforms. The CompNext1D architecture performs well in compressing hyperspectral data from multiple sensor sources with different characteristics while achieving higher reconstruction accuracy compared to state-of-the-art methods.

Nonlinear effects in hyperspectral (HS) remote sensing data, caused by shadows, varying illumination conditions, as well as by directional reflectance variations, may lead to inaccurate land cover classification. Including additional features of a simultaneously collected digital elevation model (DEM) generally improves the results. In this paper, we apply the Nonlinear Feature Normalization (NFN) to a weighted concatenation of HS channels and different sets of features derived from DEMs to improve the classification accuracy. The evaluation is performed on two data sets, where, the labeled data for one of them was derived using an interactive approach based on unsupervised classification. Using sensor data fusion and NFN transformation improved classification accuracy from a Cohens κ of 0.6 to values over 0.8.

In the last decades, hyperspectral imaging (HSI) has become an appealing field of remote sensing because of its richness in information. However, HSI suffers from redundancy and high computational complexity. To address this challenge, we present Policy Gradient Band Search - an intuitive and straightforward search strategy for dimensionality reduction of hyperspectral data. The family of policy gradient algorithms is a widely used and successful model-free approach in the reinforcement learning framework and is here applied for HSI band selection. In order to exhibit the effectiveness of our method, we evaluate our approach under consideration the spatial-spectral relationship in combination with a 3D-CNN based HSI classification network. Our approach yields satisfying empirical results in three HSI datasets, which are Pavia University, Salinas and Greding. The overall accuracy of 98.14%,96.64%,99.84% was separately accomplished on these datasets while being restricted by utilizing only four chosen bands.

Hyperspectral target detection experiments under nonideal conditions are scarce. An extensive multi-scale and multi-temporal field experiment was designed towards the goal of knowledge expansion under such circumstances. A range of camouflage materials and specific targets of interest were placed in a realistic natural environment with vegetation cover and varying illumination. In several experiments, aspects like changes in the sun position, variable moisture, and relocations of targets were analysed. Using an aircraft-based and a drone-based imaging spectrometer, the target scenarios were mapped at different daytimes. The data were radiometrically, atmospherically and geometrically processed to allow subsequent data analysis. First insights deliver promising results.

Mitigating nonlinear effects, e.g., due to shadows, variations in illumination conditions, and angular dependencies of spectral signatures is an important topic in hyperspectral remote sensing. In this paper, we apply the Nonlinear Feature Normalization on a combined data set consisting of 128 spectral bands and a weighted digital elevation model. The NFN transforms the data set to a new linear basis and by that mitigates nonlinearities. Evaluation is done by applying the Spectral Angle Mapper to the original and the NFN-transformed data. Different parameter combinations are tested to find the best classification results. Additionally, a Random Forest approach is calculated to compare the results.

Domain adaptation in remote sensing aims at the automatic knowledge transfer between a set of multitemporal and multisource images. This process is often impaired by nonlinear effects in the data, e.g., varying illumination conditions, different viewing angles, and geometry-dependent reflection. In this paper, we introduce the Nonlinear Feature Normalization (NFN), a fast and robust way to align the spectral characteristics of multiple hyperspectral data sets. NFN employs labeled training spectra for the different classes in an image to describe the corresponding underlying low-dimensional manifold structure. A linear basis for data representation is defined by arbitrary class reference vectors, and the image is aligned to the new basis in the same space. This results in samples of the same class being pulled closer together and samples of different classes pushed apart. NFN transforms the data in its original domain, preserving physical interpretability. We use the continuous invertibility of NFN to derive the NFN Alignment (NFNalign) transformation, which can be used for domain adaptation, by transforming one data set to the domain of a chosen reference. The evaluation is performed on multiple hyperspectral data sets as well as our new benchmark for multitemporal hyperspectral data. In a first step, we show that the NFN transformation successfully mitigates nonlinear effects by comparing classification of the linear Spectral Angle Mapper on original and transformed data. Finally, we demonstrate successful domain adaptation with NFNalign by applying it to the task of hyperspectral data preprocessing. The evaluation shows that our approach for alignment of multitemporal data produces high-spectral similarity and successfully allows knowledge transfer, e.g., of classifier models and training data.

At Fraunhofer IOSB, a concept for a near real-time airborne 3D mapping system for disaster management and security applications was proposed [1]. In addition to an Airborne Laser Scanner (ALS), a RGB camera facing nadir is installed on that platform. Subsequent RBG pictures are used for reconstructing urban scenes, and an ALS point cloud is used as a reference data set to evaluate this work. For in-flight applications, the implemented 3D reconstruction from overlapping nadir images should operate in near real-time. This paper focuses on computational performance and quality considerations of 3D reconstruction of urban scenes with aerial images. Different subpixel disparity levels are considered for generating 3D models. The run-time and quality for depth map generation with two methods are evaluated as well. The PatchMatch Stereo (PMS) algorithm is compared with the Semi-Global Matching (SGM), the most popular algorithm for 3D reconstruction.

Non-linear effects in hyperspectral data are the result of varying illumination conditions, angular dependencies of reflection, shadows and multiple scattering of incident light. Common classification algorithms like Spectral Angular Mapper (SAM) and Adaptive Coherence Estimator (ACE) struggle to produce good results under these conditions. In this paper, we evaluate our fast Semi-supervised Invertible Manifold Alignment, introduced in [1], on multiple commonly available hyperspectral remote sensing data sets. Additionally, we test it on our new benchmark data set for multitemporal analysis. We show that linear SAM classification on SIMA-transformed data is superior to linear classification on the original data in all cases. Also, SIMA-transformation with subsequent SAM classification produces comparable results to a multi-class Support Vector Machine (SVM), with the benefit of maintaining physical interpretability of the transformed data.

This paper suggests using color-guided depth enhancement algorithms of computer vision to improve the resolution of airborne laser scanning (ALS) point clouds for remote sensing applications. We use co-registered high resolution color images with nadir view to enhance the ALS data; and perform quantitative evaluation in form of RMSE considering the whole depth image as well as the depth discontinuities only. Investigated methods include joint bilateral filtering, Markov Random Field (MRF) optimization with first and second order smoothness terms, and anisotropic diffusion. RMSE results on discontinuities indicate that detail improvement performance of the selected methods on the depth discontinuities is not on a satisfactory level for airborne data. Anisotropic diffusion and MRF optimization are promising to provide better results with further adjustments on the smoothness terms.

Different methods for the detection for hydrocarbons in aerial hyperspectral images are analyzed in this study. The scope is to find a practical method for airborne oil spill mapping on land. Examined are Hydrocarbon index and Hydrocarbon detection index. As well as spectral reidentification algorithms, like Spectral angle mapper, in comparison to the indices. The influence of different ground coverage and different hydrocarbons was tested and evaluated. A ground measurement campaign was conducted with controlled contaminations and manual definition of ground truth data, to evaluate the performance of the detection methods. Additionally, the discriminability between wet ground and oil-contaminated ground is investigated, along with the temporal influence on oil spill detection.

Nonlinear effects in hyperspectral data complicate classification and other data analysis procedures. Transforming the data onto manifolds can help to improve the results while simultaneously reducing the dimensionality due to the high correlation among the spectral bands. Methods like ISOMAP or Locally Linear Embedding are not ideal when the data is degraded by noise. In this paper, a method is introduced to automatically generate support points for skeletonizing a high-dimensional point cloud. The skeleton is identified with multiple signal source variations of distinct materials and can be used to transform the data to improve further analysis procedures.

This paper suggests a method for automatic in-flight boresight calibration of pushbroom scanner images, using an on-line system with broadband data downlink and near realtime georeferencing of the pushbroom image data. Georeferencing accuracy may decrease during long image acquisition flights due to instable atmospheric conditions, which may lead to geometric changes in the flight platform. Orthorectification of (hyperspectral) pushbroom scanner data demands the knowledge of the extrinsic orientation parameters for every exposure. The most crucial parameters for the transformation of the pose obtained by the inertial navigation system (INS) into the projection center of the imaging sensor are the boresight angles. Utilizing a performant ray tracing algorithm and a digital elevation model (DEM), these parameters can be estimated even while flying in uneven and uninhabited areas. Tie points for solving an extended collinear equation are extracted automatically by the SURF algorithm.

In the last few years, unmixing of hyperspectral data has become of major importance. The high spectral resolution results in a loss of spatial resolution. Thus, spectra of edges and small objects are composed of mixtures of their neighboring materials. Due to the fact that supervised unmixing is impossible for extensive data sets, the unsupervised Nonnegative Matrix Factorization (NMF) is used to automatically determine the pure materials, so called endmembers, and their abundances per sample [1]. As the underlying optimization problem is nonlinear, a good initialization improves the outcome [2]. In this paper, several methods are combined to create an algorithm for fully unsupervised spectral unmixing. Major part of this paper is an initialization method, which iteratively calculates the best possible candidates for endmembers among the measured data. A termination condition is applied to prevent violations of the linear mixture model. The actual unmixing is performed by the multiplicative update from [3]. Using the proposed algorithm it is possible to perform unmixing without a priori studies and accomplish a sparse and easily interpretable solution. The algorithm was tested on different hyperspectral data sets of the sensor types AISA Hawk and AISA Eagle.

The Reference Point Method has been developed by T-Systems as an overall approach for making the complexity in developing ICT convergence solutions manageable. It is based heavily on requirements engineering at the level of business process architecture, technology neutral architecture, technology specific architecture, and telecom architecture. The integration of all these architecture levels is realized by special interfaces at each level called reference points. This method touches the areas of software architecture design and requirements engineering and emphasizes the integration of both. First, it is used to investigate the needed business processes and to get a first insight into the needed system landscape. Then, the corresponding results provide the basis for the operative planning of the ICT systems.