The successful deployment of deep learning-based techniques for autonomous systems is highly dependent on the data availability for the respective system in its deployment environment. Especially for unstructured outdoor environments, very few datasets exist for even fewer robotic platforms and scenarios. In an earlier work, we presented the German Outdoor and Offroad Dataset (GOOSE) framework along with 10000 multimodal frames from an offroad vehicle to enhance the perception capabilities in unstructured environments. In this work, we address the generalizability of the GOOSE framework. To accomplish this, we open-source the GOOSE-Ex dataset, which contains additional 5000 labeled multimodal frames from various completely different environments, recorded on a robotic excavator and a quadruped platform. We perform a comprehensive analysis of the semantic segmentation performance on different platforms and sensor modalities in unseen environments. In addition, we demonstrate how the combined datasets can be utilized for different downstream applications or competitions such as offroad navigation, object manipulation or scene completion. The dataset, its platform documentation and pre-trained state-of-the-art models for offroad perception will be made available on https://goose-dataset.de/.

The potential for deploying autonomous systems can be significantly increased by improving the perception and interpretation of the environment. However, the development of deep learning-based techniques for autonomous systems in unstructured outdoor environments poses challenges due to limited data availability for training and testing. To address this gap, we present the German Outdoor and Offroad Dataset (GOOSE), a comprehensive dataset specifically designed for unstructured outdoor environments. The GOOSE dataset incorporates 10000 labeled pairs of images and point clouds, which are utilized to train a range of state-of-the-art segmentation models on both image and point cloud data. We open source the dataset, along with an ontology for unstructured terrain, as well as dataset standards and guidelines. This initiative aims to establish a common framework, enabling the seamless inclusion of existing datasets and a fast way to enhance the perception capabilities of various robots operating in unstructured environments. This framework also makes it possible to query data for specific weather conditions or sensor setups from a database in future. The dataset, pre-trained models for offroad perception, and additional documentation can be found at https://goose-dataset.de/.

The proposed $\text{X}^{3}\text{S}\text{e}\text{g}$ approach generates model-agnostic, example-based explanations for the semantic segmentation of 3D point clouds. It retrieves the most similar 3D point sets (prototypes) as well as the most dissimilar point sets (criticism) to the spatially connected 3D point set which is to be explained. $\text{X}^{3}\text{S}\text{e}\text{g}$ comprises three methods for a holistic understanding of point-by-point class predictions: encompassing, selective, and predictive $\text{X}^{3}\text{S}\text{e}\text{g}$. Prototypes and criticism are identified from a particularly generated prototype database by combining different similarity measures. To the best of our knowledge, $\text{X}^{3}\text{S}\text{e}\text{g}$ is the first model-agnostic explainable artificial intelligence (XAI) approach providing example-based explanations for the semantic segmentation of 3D data with prototypes and criticism. It is demonstrated on RangeNet53++[1] predictions for 3D point cloud data from the SemanticKITTI dataset [2].

Fusing multiple sensors is vital for the precise online localization of mobile robots. The Extended Kalman filter is well suited for this purpose and allows to directly integrate multiple absolute measurements in the filter. However, multiple relative state measurements cannot be fused directly into the filter because they measure differences between a past state and the current state, which introduces correlations. These interdependencies can be modeled by stochastic cloning, which introduces a state augmentation by cloning the respective state estimates connected by a relative measurement. This paper investigates the impact of multiple mixed relative and absolute updates on the estimated state. An approach to feed back information from absolute updates to the cloned state by explicit cloning is presented and compared to existing implicit methods.

We propose a novel constrained planning approach for autonomous vehicles in unstructured outdoor environments. Our method enables autonomous off-road platforms to keep a predetermined track accurately, but in the same time allows the avoidance of static obstacles and dynamic objects. Two application scenarios are presented according to common transport behaviors with obstacle avoidance: Mule and Convoying. Our method is real-time integrated in a typical processing pipeline for autonomous driving in unstructured outdoor environments. It provides a constrained planning area, subsequently designated cost valley. Our cost valley keeps the trajectory of the vehicle both smooth in difficult passages and close to the desired track. The efficiency of our method is demonstrated on two autonomous platforms with a huge difference in kinematics, weight and size - a small electric platform and an off-road truck. It directly improves the behavior of autonomous vehicles, especially in critical passages.

A mobile robot is reliant on precise and robust localization and mapping for autonomous navigation. For this purpose, sensor fusion techniques are employed to combine measurements of multiple sensor data sources. The well-known Extended Kalman filter is the standard approach to integrate absolute measurements; however, multiple relative measurements, i.e., measured differences between the current system state and a past system state, cannot be directly incorporated into the filter. This paper presents a fusion algorithm for the integration of absolute and multiple relative measurements for localization and mapping of mobile robots. A novel approach exploiting concurrent stochastic cloning and smoothing is introduced for robust inclusion of additional relative measurements. The proposed fusion method is applied to perform simultaneous localization and mapping with sensor data from an IMU, a GPS, wheel odometry, and scan matching of data from a 3D LiDAR.

We present a local stereo matching method for hyperspectral camera data, allowing multiple usage of camera hardware and imaging data such as for object classification or spectral analysis and multichannel input to the correspondence problem. The matching process combines correlation-based similarity measures for pixel windows utilizing all 16 spectral channels followed by a consistency check for disparity selection. We evaluate stereo-processing methods focusing on effectiveness and runtime of the processing on a CPU and analyze parallelization possibilities. Based on the results of the evaluation on the CPU, we implement the optimized stereo matching for images with 16 channels on a graphics processing unit (GPU) utilizing the Compute Unified Device Architecture (CUDA). The parallel processing of the calculation steps to obtain the disparity image on the GPU achieves more than 27× speed up, resulting in calculation and post-processing of hyperspectral images with 8 - 13 Hz, depending on the selection of maximum disparity. The 3D reconstruction achieves a mean square error of 0.0267 m2in distance measurements from 5 - 10 m.