This work addresses the challenge of including the spatial dimension into the autoencoder models for lossy compression of different spatially independent and unknown hyperspectral datasets acquired by space-borne hyperspectral sensors. We propose two different 3D-Hybrid Convolutional Autoencoder models with increased compression rates compared to 1D methods that can compress and reconstruct hyperspectral data with arbitrary spectral dimensionality. The architecture of the first 3D-Hybrid model consists of the A1D-CAE in combination with the 2D-CAE. The second 3D-Hybrid model includes the adaptive 1D-CAE and a 3D-CAE. The evaluation of the reconstruction accuracy is measured by comparing the spectral angle and the peak signal-to-noise ratio between the original and the reconstructed data and structural similarity index measure. We show the high transferability and generalizability of our 3D-Hybrid models on different PRISMA datasets. The 3D-Hybrid model is compared with the SSCNet2Dbased on a 2D-CAE and a 3D-CAE model. The findings of this study contribute to understanding the strengths and limitations of machine learning-based compression methods for jointly compressing spectral and spatial information.

The increasing use of deep learning techniques has reduced interpretation time and, ideally, reduced interpreter bias by automatically deriving geological maps from digital outcrop models. However, accurate validation of these automated mapping approaches is a significant challenge due to the subjective nature of geological mapping and the difficulty in collecting quantitative validation data. Additionally, many state-of-the-art deep learning methods are limited to 2D image data, which is insufficient for 3D digital outcrops, such as hyperclouds. To address these challenges, we present Tinto, a multi-sensor benchmark digital outcrop dataset designed to facilitate the development and validation of deep learning approaches for geological mapping, especially for non-structured 3D data like point clouds. Tinto comprises two complementary sets: 1) a real digital outcrop model from Corta Atalaya (Spain), with spectral attributes and ground-truth data, and 2) a synthetic twin that uses latent features in the original datasets to reconstruct realistic spectral data (including sensor noise and processing artifacts) from the ground-truth. The point cloud is dense and contains 3,242,964 labeled points. We used these datasets to explore the abilities of different deep learning approaches for automated geological mapping. By making Tinto publicly available, we hope to foster the development and adaptation of new deep learning tools for 3D applications in Earth sciences. The dataset can be accessed through this link: https://doi.org/10.14278/rodare.2256.

This work addresses the challenge of transferability of autoencoder models for lossy compression of different spatially independent and unknown hyperspectral datasets acquired from different space sensor platforms. We propose an adaptive 1D convolutional autoencoder architecture that can compress and recover spectral signatures with different numbers of bands. We demonstrate the transferability of the 1D CAE to different sensors by applying different unknown hyperspectral datasets acquired by different sensor platforms. The evaluation of the reconstruction accuracy is measured by comparing the spectral angle and the signal-to-noise ratio between the original and the reconstructed data. We show the high transferability and generalizability of our A1D-CAE model for compression rates cR= 4 on different datasets from the satellite-based PRISMA, DESIS, EnMap and HYPSO-1 sensors. The results show that the proposed A1D-CAE architecture is capable of processing hyperspectral data from multiple sensor sources with different characteristics while achieving high reconstruction accuracy.

The growing availability of hyperspectral remote sensing data, specifically from the new hyperspectral satellite missions, requires efficient data compression due to limitations in bandwidth and available storage space while simultaneously preserving the spectral characteristics. Machine learning approaches are a powerful way to address this challenge, but they are usually tailored to only work on one specific sensor. This work addresses the challenge of transferability of autoencoder models for lossy compression of spatially independent and unknown hyperspectral datasets acquired from different sensor platforms. We propose the CompNext1D, an advanced multi-stage adaptive network based on the architecture of the A1D-CAE. The characteristic of the A1D-CAE allows pre-training on a large dataset with wide spectral variability and transferability to other sensor data, e.g., with potentially limited data availability. The compression performance of the CompNext1D is enhanced by using image statistics and shows a high degree of transferability to unknown spectral signatures. We evaluate the reconstruction accuracy with three experiments of increasing complexity. The evaluation is based on the reconstruction accuracy using the SA, SNR, PSNR, and SSIM metrics, and the results are compared to other learning-based lossy compression techniques.We demonstrate the high transferability and generalizability of our A1D-CAE and CompNext1D for compression rates fromcR= 4 tocR≈ 100 on hyperspectral data from different sensors and carrier platforms. The CompNext1D architecture performs well in compressing hyperspectral data from multiple sensor sources with different characteristics while achieving higher reconstruction accuracy compared to state-of-the-art methods.

We present a novel method to reconstruct a spectral central view and its aligned disparity map from spatio-spectrally coded light fields. Since we do not reconstruct an intermediate full light field from the coded measurement, we refer to this as principal reconstruction. We show that the direct estimation is superior to a full light field reconstruction and subsequent disparity estimation. The coded light fields correspond to those captured by a light field camera in the unfocused design with a spectrally coded microlens array. In this application, the spectrally coded light field camera can be interpreted as a single-shot spectral depth camera. We investigate several multi-task deep learning methods and propose a new auxiliary loss-based training strategy to enhance the reconstruction performance. The results are evaluated using a synthetic as well as a new real-world spectral light field dataset that we captured using a custom-built camera. The results are compared to state-of-the art compressed sensing reconstruction and disparity estimation. We achieve a high reconstruction quality for both synthetic and real-world coded light fields. The disparity estimation quality is on par with or even outperforms state-of-the-art disparity estimation from uncoded RGB light fields.

The proposed $\text{X}^{3}\text{S}\text{e}\text{g}$ approach generates model-agnostic, example-based explanations for the semantic segmentation of 3D point clouds. It retrieves the most similar 3D point sets (prototypes) as well as the most dissimilar point sets (criticism) to the spatially connected 3D point set which is to be explained. $\text{X}^{3}\text{S}\text{e}\text{g}$ comprises three methods for a holistic understanding of point-by-point class predictions: encompassing, selective, and predictive $\text{X}^{3}\text{S}\text{e}\text{g}$. Prototypes and criticism are identified from a particularly generated prototype database by combining different similarity measures. To the best of our knowledge, $\text{X}^{3}\text{S}\text{e}\text{g}$ is the first model-agnostic explainable artificial intelligence (XAI) approach providing example-based explanations for the semantic segmentation of 3D data with prototypes and criticism. It is demonstrated on RangeNet53++[1] predictions for 3D point cloud data from the SemanticKITTI dataset [2].

Data fusion in multi-sensor systems requires an accurate calibration of the sensors. To this end, the sensor data itself can be registered, which evades often inaccurate, manual measurements on the sensor setup. This work proposes a flexible registration framework for the calibration of multi-sensor systems in unstructured environments, denoted Unstructured Cross-Source Registration (UCSR). In unstructured environments, the registration of sensor data presents a major challenge due to the absence of structure, flat surfaces, and clearly separated objects. Cross-source data, captured with different types of sensors, presents differences in scale, measurement density, accuracy, noise, and outlier characteristics, which poses an additional challenge in registration. At present, three methods for the cross-source registration are included in UCSR. Each method is evaluated independently on real-world 2D and 3D data. To achieve a stable calibration, UCSR combines the registration results of the individual methods according to their accuracy and robustness. UCSR does neither require special calibration objects nor human intervention. Cross-source 2D and 3D sensor data is captured with a mobile robotic off-road platform. UCSR is able to register sensor data with a decalibration of 2.0 m along and 20° around each axis. UCSR achieves a mean accuracy of 5.1 cm and 0.956° on 2D and 3D data from unstructured environments.

In order to make convolutional neural networks (CNNs) usable on smaller or mobile devices, it is necessary to reduce the computing, energy and storage requirements of these networks. One can achieved this by a fixed-point quantization of weights and activations of a CNN, which are usually represented by 32-bit floating-point. In this paper, we present an adaption of convolutional and fully connected layers in order to obtain a high usage of the given value range of activations and weights. Therefore, we introduce scaling factors obtained by moving average to limit the weights and activations. Our model, quantized to 8 bit, outperforms the 7-layer baseline model from which it is derived and the naive quantization by several percentage points. Our method does not require any additional operations in the inference and both the weights and activations have a fixed radix point.

Remaining useful life estimation is a research topic of high relevance in the area of structural mechanics. To predict the remaining useful lifetime of a motor, domain experts commonly employ physical simulations based on 3D-CAD models. However, this process is laborious and in many cases no 3D-CAD model is available. Also, setting up a simulation might require substantial efforts or might even be infeasible. This article focuses on the machine learning based estimation of the remaining useful life of unknown, derived motor types of an electric motor class based on simulations of known motor types, as well as data sheets and measurements. In particular, we propose the hybrid fusion method moSAIc that allows to transfer the knowledge inherent in physical degradation models of motors to unknown instances. Our experiments show that moSAIc outperforms other state-of-the-art methods by a large margin in terms of both accuracy and robustness. Furthermore, compared to purely data-driven methods such as neural networks, moSAIc is explainable allowing domain experts to understand the reason for the predictions.

Online condition monitoring services and predictive maintenance are becoming more and more a key for system operators to extend the system lifetime and detect faults in early stages. Therefore, system manufactures need to efficiently provide system operators so-called digital twins which can be executed during operation and give the system operator an impression of the health state of the system. Industrial Internet of Things (IIoT) platforms are enablers for such services and provide new possibilities to interact with the system running in the field. Furthermore, the traditional dashboard are becoming obsolete as user interface and are replaced by novel solutions that let the system operator experience the system health state. For example, health estimation and condition monitoring of electric motors is a topic of high interest nowadays. This article addresses an application which acquires machine data, processes it on an IIoT platform to get the system health and visualizes the results online in an augmented reality user interface.

Due to decreasing hardware prices, machine learning is becoming increasingly interesting for industrial applications such as automatic visual inspection (AVI). This paper presents a metaheuristic approach to the automatic generation of a well suited convolutional neural network (CNN) based on differential evolution. This makes it possible to find a suitable architecture of a CNN for a given task with little prior knowledge. Another aim is to reduce the resources needed in the inference as much as possible. Therefore, we choose a function that considers both the accuracy and the resources used to measure the fitness of a CNN. For typical industrial datasets, we obtain CNNs with an accuracy of more than 98 % on average within relatively short processing time.

The combination of different modeling approaches has been applied to create so-called Grey-Box-Models for a few decades. But due to increasing data availability provided by IoT-devices, new possibilities regarding data-driven modeling arise. The purpose of this article is to develop a concept which is able to combine data-driven models created during operation with such respective theoretical models created during the design phase. The model combination should result in an added value for the operator at shop floor in the shape of real-time-capable simulation models. Initially, the different modeling methods as well as the way they can be combined are introduced. Afterwards, a concept for training and operation will be presented. The article ends with a first example demonstrating the potential of the approach.

Despite its age the inspection of specular surfaces is still a topic of ongoing research. While sensory approaches to inspect such surfaces based on deflectometry are increasingly used in practice, the evaluation techniques using the acquired signals (images and reconstruction results) are often not sufficient. This work addresses the challenge of detecting defects with different characteristics on specular surfaces by using robust multiscale detection and classification. In order to process the signals obtained by deflectometry efficiently in all relevant scales, a method for generating an optimized biorthogonal wavelet filter bank with strong correlation to any number of anomaly classes is proposed. The filter bank is optimized for each defect class to obtain a sparse scale space representation. In addition a Bayesian classification approach is presented to classify defects like dents and pimples directly in the scale space.

In this contribution, image processing techniques are investigated to robustly segment faint striation patterns from mainly isotropic background areas. Whereas well-known procedures to determine local anisotropy can be applied directly to single images, this paper describes an alternative strategy that uses an illumination series. The series is obtained with a spot illumination whose azimuth is varied systematically. The technique is based on the characteristics of the local contrast with respect to the azimuth of the illumination: Whereas pronounced maxima of the local contrast can be observed when the striae are illuminated perpendicularly, an isotropic background texture shows less distinct maxima for random azimuth angles. The qualification of the approach is demonstrated on the segmentation of faint tool marks in forensic science. Experimental results show that the methodology ensures the correct segmentation of such marks.