High-definition (HD) maps are crucial for autonomous vehicles, but their creation and maintenance is very costly. This motivates the idea of online HD map construction. To provide a continuous large-scale stream of training data, existing HD maps can be used as labels for onboard sensor data from consumer vehicle fleets. However, compared to current, well curated HD map perception datasets, this fleet data suffers from localization errors, resulting in distorted map labels. We introduce three kinds of localization errors, Ramp, Gaussian, and Perlin noise, to examine their influence on generated map labels. We train a variant of MapTRv2, a state-of-the-art on-line HD map construction model, on the Argoverse 2 dataset with various levels of localization errors and assess the degradation of model performance. Since localization errors affect distant labels more severely, but are also less significant to driving performance, we introduce a distance-based map construction metric. Our experiments reveal that localization noise affects the model performance significantly. We demonstrate that errors in heading angle exert a more substantial influence than position errors, as angle errors result in a greater distortion of labels as distance to the vehicle increases. Furthermore, we can demonstrate that the model benefits from nondistorted ground truth (GT) data and that the performance decreases more than linearly with the increase in noisy data. Our study additionally provides a qualitative evaluation of the extent to which localization errors influence the construction of HD maps.

Teleoperation enables remote human support of automated vehicles in scenarios where the automation is not able to find an appropriate solution. Remote assistance concepts, where operators provide discrete inputs to aid specific automation modules like planning, is gaining interest due to its reduced workload on the human remote operator and improved safety. However, these concepts are challenging to implement and maintain due to their deep integration and interaction with the automated driving system. In this paper, we propose a solution to facilitate the implementation of remote assistance concepts that intervene on planning level and extend the operational design domain of the vehicle at runtime. Using arbitration graphs, a modular decision-making framework, we integrate remote assistance into an existing automated driving system without modifying the original software components. Our simulative implementation demonstrates this approach in two use cases, allowing operators to adjust planner constraints and enable trajectory generation beyond nominal operational design domains.

Autonomous vehicles rely on detailed and accurate environmental information to operate safely. High definition (HD) maps offer a promising solution, but their high maintenance cost poses a significant barrier to scalable deployment. This challenge is addressed by online HD map construction methods, which generate local HD maps from live sensor data. However, these methods are inherently limited by the short perception range of onboard sensors. To overcome this limitation and improve general performance, recent approaches have explored the use of standard definition (SD) maps as prior, which are significantly easier to maintain. We propose SDTagNet, the first online HD map construction method that fully utilizes the information of widely available SD maps, like OpenStreetMap, to enhance far range detection accuracy. Our approach introduces two key innovations. First, in contrast to previous work, we incorporate not only polyline SD map data with manually selected classes, but additional semantic information in the form of textual annotations. In this way, we enrich SD vector map tokens with NLP-derived features, eliminating the dependency on predefined specifications or exhaustive class taxonomies. Second, we introduce a point-level SD map encoder together with orthogonal element identifiers to uniformly integrate all types of map elements. Experiments on Argoverse 2 and nuScenes show that this boosts map perception performance by up to +5.9 mAP (+45%) w.r.t. map construction without priors and up to +3.2 mAP (+20%) w.r.t. previous approaches that already use SD map priors.

The Large-scale 3D reconstruction, texturing and semantic mapping are nowadays widely used for automated driving vehicles, virtual reality and automatic data generation. However, most approaches are developed for RGB-D cameras with colored dense point clouds and not suitable for large-scale outdoor environments using sparse LiDAR point clouds. Since a 3D surface can be usually observed from multiple camera images with different view poses, an optimal image patch selection for the texturing and an optimal semantic class estimation for the semantic mapping are still challenging.To address these problems, we propose a novel 3D reconstruction, texturing and semantic mapping system using LiDAR and camera sensors. An Adaptive Truncated Signed Distance Function is introduced to describe surfaces implicitly, which can deal with different LiDAR point sparsities and improve model quality. The from this implicit function extracted triangle mesh map is then textured from a series of registered camera images by applying an optimal image patch selection strategy. Besides that, a Markov Random Field-based data fusion approach is proposed to estimate the optimal semantic class for each triangle mesh. Our approach is evaluated on a synthetic dataset, the KITTI dataset and a dataset recorded with our experimental vehicle. The results show that the 3D models generated using our approach are more accurate in comparison to using other state-of-the-art approaches. The texturing and semantic mapping achieve also very promising results.

While complete localization approaches are widely studied in the literature, their data association and data representation subprocesses usually go unnoticed. However, both are a key part of the final pose estimation. In this work, we present DA-LMR (Delta-Angle Lane Marking Representation), a robust data representation in the context of localization approaches. We propose a representation of lane markings that encodes how a curve changes in each point and includes this information in an additional dimension, thus providing a more detailed geometric structure description of the data. We also propose DC-SAC (Distance-Compatible Sample Consensus), a data association method. This is a heuristic version of RANSAC that dramatically reduces the hypothesis space by distance compatibility restrictions. We compare the presented methods with some state-of-the-art data representation and data association approaches in different noisy scenarios. The DA-LMR and DC-SAC produce the most promising combination among those compared, reaching 98.1 % in precision and 99.7% in recall for noisy data with 0.5 m of standard deviation.

In this paper, we present TEScalib, a novel extrinsic self-calibration approach of LiDAR and stereo camera using the geometric and photometric information of surrounding environments without any calibration targets for automated driving vehicles. Since LiDAR and stereo camera are widely used for sensor data fusion on automated driving vehicles, their extrinsic calibration is highly important. However, most of the LiDAR and stereo camera calibration approaches are mainly target-based and therefore time consuming. Even the newly developed targetless approaches in last years are either inaccurate or unsuitable for driving platforms. To address those problems, we introduce TEScalib. By applying a 3D mesh reconstruction-based point cloud registration, the geometric information is used to estimate the LiDAR to stereo camera extrinsic parameters accurately and robustly. To calibrate the stereo camera, a photometric error function is builded and the LiDAR depth is involved to transform key points from one camera to another. During driving, these two parts are processed iteratively. Besides that, we also propose an uncertainty analysis for reflecting the reliability of the estimated extrinsic parameters. Our TEScalib approach evaluated on the KITTI dataset achieves very promising results.

Currently, there is a gap in motion planning approaches. On the one hand, there are optimization-based motion planning techniques which can guarantee safety and feasibility, but are either slow and cooperative or fast and uncooperative. On the other hand, there are learned approaches that are fast and cooperative, but cannot give these desirable guarantees.We propose to combine model predictive contouring control (MPCC) with sophisticated collision avoidance formulations to bridge this gap. By optimizing the total utility of all traffic participants, a cooperative, safe, and feasible trajectory can be planned in real time.Examination of various collision avoidance constraints allows to obtain considerate trajectories while preserving real-time capabilities. A novel inter-stage constraint formulation allows to introduce time-based distance measures in time-discretized MPC formulations.We evaluate the resulting motion planner in various scenarios, comparing two state-of-the-art solvers.

When detecting objects, depth sensors are not always available, requiring 3D object detection from monocular images. However, for many object classes, datasets with 3D annotations are missing. Recent monocular 3D object detection methods lack the semantic diversity needed for autonomous systems, because of missing 3D ground truth data for static classes such as poles and traffic lights. To overcome this gap we combine a large scale dataset for 2D object detection, with an unlabeled dataset containing depth measurements. We lift 2D object detections of the depth dataset into the 3D domain, associating detections with corresponding depth values. This leverages 2D annotated datasets to enable semantically rich 3D object detection, without extra labelling effort. We train an object detection model with mixed batches and evaluate it comparing the predicted depth with the projected centerpoint depth of cars manually annotated in 3D space. The result is a monocular object detector that can predict 3D positions of up to 37 static and dynamic object classes from camera only.

While the automatic creation of maps for localization is a widely tackled problem, the automatic inference of higher layers of HD maps is not. Additionally, approaches that learn from maps require richer and more precise landmarks than currently available.In this work, we fuse semantic detections from a monocular camera with depth and orientation estimation from lidar to automatically detect, track and map parametric, semantic map elements. We propose the use of tailored representations that are minimal in the number of parameters, making the map compact and the estimation robust and precise enough to enable map inference even from single frame detections. As examples, we map traffic signs, traffic lights and poles using upright rectangles and cylinders.After robust multi-view optimization, traffic lights and signs have a mean absolute position error of below 10 cm, extent estimates are below 5 cm and orientation MAE is below 6◦. This proves the suitability as automatically generated, pixel-accurate ground truth, reducing the task of ground truth generation from tedious 3D annotation to a post-processing of misdetections.

High definition (HD) maps can fail by becoming outdated. To still use them safely for automated driving, they need to be verified or updated, both requiring methods for change detection. We propose two significant improvements for HD map change detection that do not require a highly accurate localization prior as localization quickly fails or cannot be trusted in an outdated map. Given a very coarse localization prior, we group stored or measured map features in spatially and semantically separable feature groups. These feature groups are not only intuitive, like the sequence of leftmost dashed lane markings, but changes are also highly correlated within them. The first contribution improves the way internal consistency of each feature group is assured by using boosted classification trees. Additionally, a mutual evaluation scheme is added for all seemingly unchanged feature groups. Always one feature group is used for localization by feature alignment while each other group's alignment is checked for compatibility. Two voting schemes are presented that allow a more or less sensitive change detection on the level of proposed groups. In contrast to almost all other approaches, our approach allows to use still valid parts of the map for automated driving and to update the changed parts. We evaluate our approach on a previously published map verification dataset [1], showing that the number of undetected map changes can be reduced by up to 31 % compared to state of the art using boosted classification trees, at the same time reducing false positive rates by up to 50 %. The additional mutual evaluation step is able to uncover a whole category of previously undetectable changes and reduces undetected changes by an extra 15 %.

Easy, yet robust long-term localization is still an open topic in research. Existing approaches require either dense maps, expensive sensors, specialized map features or proprietary detectors.We propose using semantic segmentation on a monocular camera to localize directly in a HD map as used for automated driving. This combines lightweight, yet powerful HD maps with the simplicity of monocular vision and the flexibility of neural networks.The major challenges arising from this combination are data association and robustness against misdetections. Association is solved efficiently by applying distance transform on binary per-class images. This provides not only a fast lookup table for a smooth gradient as needed for pose-graph optimization, but also dynamic association by default.A sliding-window pose graph optimization combines single image detections with vehicle odometry, smoothing results and helping overcome even misclassifications in consecutive frames.Evaluation against a highly accurate 6D visual localization shows that our approach can achieve accuracy levels as required for automated driving, being one of the most lightweight and flexible methods to do so.

High definition (HD) maps have proven to be a necessary component for safe and comfortable automated driving (AD) [1]. Naïvely verifying HD maps requires an accurate localization prior in order to correctly associate measurements with map data. In periodic environments, such as highways, localization results are often ambiguous - in particular in longitudinal direction. To still be able to verify an HD map, we propose the use of quasi-continuous 1D signals that can be computed without pointwise association. These signals can be chosen to change significantly when the map has changed while they only change rarely or slowly along the road, making them robust against localization errors. A spatio-semantic clustering yields intuitive groups of map features. These groups are then ordered using a robust projection approach, yielding quasi-continuous 1D signals. Such signals can be computed for map and measurement data and their comparison allows detecting road changes. The purposeful design of the signals and their computation only requires lane-level lateral localization and a coarse longitudinal prior, vastly relaxing the requirements on prior localization results compared to the current state of the art. With four example signals, we demonstrate the effectiveness of our approach on a map verification dataset [2], detecting between 49 % and 98 % of all changed features at false alarm rates usually below 15 %. Detecting changes per feature allows to still use unchanged features for AD functions. When omitting this ability and aggregating all features, 98 % of all changed road sections can be detected successfully.successfully.