Automotive self-localization is an essential task for any automated driving function. This means that the vehicle has to reliably know its position and orientation with an accuracy of a few centimeters and degrees, respectively. This paper presents a radar-based approach to self-localization, which exploits fully polarimetric scattering information for robust landmark detection. The proposed method requires no input from sensors other than radar during localization for a given map. By association of landmark observations with map landmarks, the vehicle’s position is inferred. Abstract point- and line-shaped landmarks allow for compact map sizes and, in combination with the factor graph formulation used, for an efficient implementation. Evaluation of extensive real-world experiments in diverse environments shows a promising overall localization performance of 0.12m RMS absolute trajectory and 0.43° RMS heading error by leveraging the polarimetric information. A comparison of the performance of different levels of polarimetric information proves the advantage in challenging scenarios.

This paper presents a robust detector of point-shaped landmarks by leveraging scattering information from polarimetric covariance radar gridmaps. Reliable landmarks are an essential part of any feature-based vehicle self-localization required for highly automated and autonomous driving. A polarimetric distance measure is used to extract cells with scattering behavior that is uniquely distinct from its neighborhood. Experiments on real-world data are evaluated and results from a single-, dual- and full-polarization utilization are compared. By association with a high definition map, a comparison of the suitability of different types of landmarks such as poles, guideposts and tree trunks for localization applications is made.

Conventional sensor systems record information about directly visible objects, whereas occluded scene components are considered lost in the measurement process. Non-line-of-sight (NLOS) methods try to recover such hidden objects from their indirect reflections - faint signal components, traditionally treated as measurement noise. Existing NLOS approaches struggle to record these low-signal components outside the lab, and do not scale to large-scale outdoor scenes and high-speed motion, typical in automotive scenarios. In particular, optical NLOS capture is fundamentally limited by the quartic intensity falloff of diffuse indirect reflections. In this work, we depart from visible-wavelength approaches and demonstrate detection, classification, and tracking of hidden objects in large-scale dynamic environments using Doppler radars that can be manufactured at low-cost in series production. To untangle noisy indirect and direct reflections, we learn from temporal sequences of Doppler velocity and position measurements, which we fuse in a joint NLOS detection and tracking network over time. We validate the approach on in-the-wild automotive scenes, including sequences of parked cars or house facades as relay surfaces, and demonstrate low-cost, real-time NLOS in dynamic automotive environments.

Radar-based road user detection is an important topic in the context of autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to refine during subsequent signal processing. On the other hand, a new sensor generation is waiting in the wings for its application in this challenging field. In this article, two sensors of different radar generations are evaluated against each other. The evaluation criterion is the performance on moving road user object detection and classification tasks. To this end, two data sets originating from an off-the-shelf radar and a high resolution next generation radar are compared. Special attention is given on how the two data sets are assembled in order to make them comparable. The utilized object detector consists of a clustering algorithm, a feature extraction module, and a recurrent neural network ensemble for classification. For the assessment, all components are evaluated both individually and, for the first time, as a whole. This allows for indicating where overall performance improvements have their origin in the pipeline. Furthermore, the generalization capabilities of both data sets are evaluated and important comparison metrics for automotive radar object detection are discussed. Results show clear benefits of the next generation radar. Interestingly, those benefits do not actually occur due to better performance at the classification stage, but rather because of the vast improvements at the clustering stage.

Reliable tracking of road users plays a critical part on the way to safe automated driving. In this paper, a machine learning based tracking approach on radar data is presented utilizing the radar target point clouds from multiple time steps as input to detect road users and to predict their tracking information. The detection and tracking of objects is achieved by applying a combination of known feature extractors from lidar and camera detection tasks. The generated feature maps are used as input to two branches - one branch for detection and one for tracking. In experiments on an extensive real-world radar data set, the proposed model achieves promising results in tracking performance compared to a basic clustering and a classification assisted tracking approach.

Automotive radars are becoming increasingly popular sensors for vehicle self-localization tasks because of their robustness and affordability. Newly available polarimetric sensors provide rich information about the occurring scattering mechanisms and thus enhance the uniqueness of landmarks leading to increased reliability. This paper presents a novel approach to incorporate the polarimetric information into a gridmap based on covariance matrices. Thereby, knowledge about up to three different scattering mechanisms per cell can be preserved. Such a discrimination is essential to account for the changing scattering information caused by varying viewing angles with respect to the landmark, which result from the vehicle passing by on the one hand and different mounting positions in a multi-sensor fusion setup on the other. To demonstrate suitability of the covariance representation, an evaluation of the proposed approach based on real-world experiments is presented. A substantial improvement in landmark recognition performance in difficult situations is achieved by taking advantage of the polarimetric covariance information.

Radar-based road user classification is an important yet still challenging task towards autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to recover by subsequent signal processing. In this article, classifier ensembles originating from a one-vs-one binarization paradigm are enriched by one-vs-all correction classifiers. They are utilized to efficiently classify individual traffic participants and also identify hidden object classes which have not been presented to the classifiers during training. For each classifier of the ensemble an individual feature set is determined from a total set of 98 features. Thereby, the overall classification performance can be improved when compared to previous methods and, additionally, novel classes can be identified much more accurately. Furthermore, the proposed structure allows to give new insights in the importance of features for the recognition of individual classes which is crucial for the development of new algorithms and sensor requirements.

Ahstract- This paper presents a novel discussion on comparison of common extended object tracking methods for estimating target-extension ellipses based on real world road level traffic data and a unique presentation on different target level signatures obtained for various object types such as cars, pedestrians and bicyclist obtained using automotive multi-mode radar network and 4x-Iayer automotive Iidar. The most commonly used extended object tracking methods are briefly introduced. In addition, measurement profiles of road users, car, cyclists, and pedestrians are investigated to compare each model and its assumptions with the real data. The obtained information can be further used to define the appropriate object specific and sensor specific measurement model. The measurement distribution is discussed in detail and compared with the models' assumptions. Extended object tracking is performed on the obtained data and performance analysis is carried out using high resolution ground truth. The evaluation is carried out separately on the radar and lidar data on exemplary traffic scenes. As a final step, a track-to-track intersection fusion approach is evaluated on the same data set to find out the different information gains. The relation between measurements and track behavior as well as other influences are discussed.

Semantic segmentation on radar point clouds is a new challenging task in radar data processing. We demonstrate how this task can be performed and provide results on a large data set of manually labeled radar reflections. In contrast to previous approaches where generated feature vectors from clustered reflections were used as an input for a classifier, now the whole radar point cloud is used as an input and class probabilities are obtained for every single reflection. We thereby eliminate the need for clustering algorithms and manually selected features.

The classification of individual traffic participants is a complex task, especially for challenging scenarios with multiple road users or under bad weather conditions. Radar sensors provide an - with respect to well established camera systems - orthogonal way of measuring such scenes. In order to gain accurate classification results, 50 different features are extracted from the measurement data and tested on their performance. From these features a suitable subset is chosen and passed to random forest and long short-term memory (LSTM) classifiers to obtain class predictions for the radar input. Moreover, it is shown why data imbalance is an inherent problem in automotive radar classification when the dataset is not sufficiently large. To overcome this issue, classifier binarization is used among other techniques in order to better account for underrepresented classes. A new method to couple the resulting probabilities is proposed and compared to others with great success. Final results show substantial improvements when compared to ordinary multiclass classification.

Radar has a long tradition in driver assistance systems. One of its advantages is its robustness against certain light and weather conditions. It is usually used to track surrounding objects and to detect objects which can be utilized for assistance functions, e.g. for emergency braking. However for autonomous driving, a deeper understanding of the vehicle's surroundings is necessary. In this paper we show that semantic knowledge can be obtained from a radar grid by classifying the containing objects on the cell level. This allows for omitting a prior object extraction and results directly in a semantic radar grid. We show on a data set recorded with four radar sensors that this approach produces good results distinguishing cars and other objects.

This paper presents new methods for the representation of a vehicle's contour by an oriented rectangle, also known as the bounding box. The parameters of this bounding box are originally modeled probabilistically by a single multivariate Gaussian distribution. This approach incorporates the sensor uncertainties, where the problem of estimating the parameters of this distribution from range measurements is addressed. Additionally, a transformation of the parameters into the measurement space is introduced. This representation is used to perform probabilistic updates by new measurements. The proposed method can handle strong parameter changes which might be affected by object occlusion. Experiments on real-world data demonstrate the robustness and accuracy of the probabilistic approach integrated in a tracking framework incorporating the Doppler measurements of automotive radars and laser measurements.

In an urban environment, one of the key challenges remains to be the reliable estimation of the other traffic participants' motion state. Due to the highly nonlinear motions in city traffic, an instant and precise estimation of heading direction, velocity, and, particularly, yaw rate is required. Radar sensors are well suited for this task due to their robustness to environmental influences and direct measurement of the radial (Doppler) velocity. High-resolution radars receive multiple reflections from an extended object. In comparison to state-of-the-art approaches, not only is the Doppler velocity of a single reference point taken into account, but also is the distribution of the Doppler velocity across the vehicle analyzed. The velocity profile is derived with characteristic features and a corresponding sample covariance. These are fused into an unscented Kalman filter, resulting in a significant accuracy improvement and a reduction in the latency of the filter to almost zero during a change in motion or initialization. This yields a great improvement in determining the trajectories of potential critical objects, increasing the time to avoid collisions. Furthermore, the approach enables simultaneous identification of the rotation center of the object, which is essential for the tracking of highly dynamic maneuvers. All approaches were implemented and evaluated on a large experimental data set using highly precise reference systems as ground truth. The results show an impressive improvement in the accuracy of the yaw rate estimation of a factor of 3-4 compared with state-of-the-art approaches in a dynamic scenario.

An ego-motion estimation method based on the spatial and Doppler information obtained by an automotive radar is proposed. The estimation of the motion state vector is performed in a density-based framework. Compared to standard vehicle odometry the approach is capable to estimate the full two dimensional motion state with three degrees of freedom. The measurement of a Doppler radar sensor is represented as a mixture of Gaussians. This mixture is matched with the mixture of a previous measurement by applying the appropriate egomotion transformation. The parameters of the transformation are found by the optimization of a suitable join metric. Due to the Doppler information the method is very robust against disturbances by moving objects and clutter. It provides excellent results for highly nonlinear movements. Real world results of the proposed method are presented. The measurements are obtained by a 77GHz radar sensor mounted on a test vehicle. A comparison using a high-precision inertial measurement unit with differential GPS support is made. The results show a high accuracy in velocity and yaw-rate estimation.

Grid map registration is an important field in mobile robotics. Applications in which multiple robots are involved benefit from multiple aligned grid maps as they provide an efficient exploration of the environment in parallel. In this paper, a normal distribution transform (NDT)-based approach for grid map registration is presented. For simultaneous mapping and localization approaches on laser data, the NDT is widely used to align new laser scans to reference scans. The original grid quantization-based NDT results in good registration performances but has poor convergence properties due to discontinuities of the optimization function and absolute grid resolution. This paper shows that clustering techniques overcome disadvantages of the original NDT by significantly improving the convergence basin for aligning grid maps. A multi-scale clustering method results in an improved registration performance which is shown on real world experiments on radar data.

The estimation of the ego-vehicle's motion is a key capability for advanced driving assistant systems and mobile robot localization. The following paper presents a robust algorithm using radar sensors to instantly determine the complete 2D motion state of the ego-vehicle (longitudinal, lateral velocity and yaw rate). It evaluates the relative motion between at least two Doppler radar sensors and their received stationary reflections (targets). Based on the distribution of their radial velocities across the azimuth angle, non-stationary targets and clutter are excluded. The ego-motion and its corresponding covariance matrix are estimated. The algorithm does not require any preprocessing steps such as clustering or clutter suppression and does not contain any model assumptions. The sensors can be mounted at any position on the vehicle. A common field of view is not required, avoiding target association in space. As an additional benefit, all targets are instantly labeled as stationary or non-stationary.

For autonomous parking applications to become possible, knowledge about the parking environment is required. Therefore, a real-time algorithm for detecting parked vehicles from radar data is presented. These data are first accumulated in an occupancy grid from which objects are detected by applying techniques borrowed from the computer vision field. Two random forest classifiers are trained to recognize two categories of objects: parallel-parked vehicles and cross-parked vehicles. Performances of the classifiers are evaluated as well as the capacity of the complete system to detect parked vehicles in real world scenarios.

Based on high-resolution radars a new approach for determining the full 2D-motion state (yaw rate, longitudinal and lateral speed) of an extended rigid object in a single measurement is proposed. The system does not rely on any model assumptions and is independent of the exact position, expansion and orientation of the object. In comparison to related methods it is not based on temporal filtering, e.g. a Kalman Filter. These methods are subject to an initialization phase and depend heavily on compliance of the underlying dynamic model. In contrast to temporal filtering, the proposed approach reduces the time to react to critical situations that occur in many safety and advanced driving assistance applications. This paper analyzes the velocity profile (radial velocity over azimuth angles) of the object received by two Doppler radar sensors. The approach can handle white noise and systematic variations (e.g. micro-Doppler of wheels) in the signal. The proposed system is applied to predict the driving path of traffic participants. Measurement results are presented for a set-up with two 77 GHz automotive radar sensors.

Robust tracking of extended objects plays a major role in research on highly automated driving applications and advanced driver assistance systems. This paper proposes a new approach to estimate the extension of dynamic objects based on object-local occupancy grid maps. This enables estimating free-formed object shapes while being robust against errors coming from, e.g., incorrect segmentation or association. Its benefit is shown based on 4-layer laser scanner sensor data and is evaluated against a ground truth based on a D-GPS system fused with a highly accurate IMU.

Tracking of extended targets in high definition 360 degree 3D-LIDAR (Light Detection and Ranging) measurements is a challenging task. It is a key component in robotic applications and is relevant to collision avoidance and autonomous driving. This paper presents a robust method to determine the 3D outline contour of vehicles in disordered 3D-LIDAR measurements while using several geometrical vehicle-specific constraints. In addition, the 3D outline contour contains information on the local reliability of the contour. A weighted registration approach allows calculating the velocity of consecutive 3D outline contours directly. The approach is tested with real sensor data. A robot car equipped with an inertial measurement unit serves as ground truth.

This work presents three imaging radar sensor approaches to estimate road courses, needed by intelligent vehicle systems such as active cruise control or collision avoidance. Two of the approaches use gridmap data. A gridmap integrates each measurement in a chronological order. The third approach analyzes moving objects ahead of the ego vehicle. One approach has been published previously, the other two are new. A range estimation is necessary on country roads and completes each approach. All approaches are evaluated using a huge dataset of country roads. The driven trajectory is taken as ground truth for the evaluation. The advantages and disadvantages are determined for each approach. The results show the new approach based on gridmap data performs up to 78% better than the known one. The other new approach using moving objects as input information yields estimations which are about three times more accurate than the ones from the known approach.

Automotive radar sensors are commonly used for providing environment information required by driver assistance systems. Although they show a good performance in measuring the distance and the speed of other objects even under poor weather conditions, they suffer from the shortcoming of a missing resolution in elevation. Consequently bridges crossing the lane of the ego vehicle may look similar to stationary obstacles. This contribution shows a bridge identification algorithm based on the interference pattern resulting from the multipath propagation of the radar wave. During the approaching to bridges or stationary obstacles, the pattern leads to a variation in the backscattered power from the object. This variation can be used to make statements about the object height position above the road. Results calculated from real scanning radar sensor data show the usability in real traffic scenarios.

This work presents three different quality functions for road course estimation using an imaging radar sensor. The quality functions work on gridmap data. A gridmap integrates each measurement in chronological order. Range estimation is found out to be necessary on country roads and a solution is introduced. All quality functions are evaluated using a huge set of data consisting of highways and country roads. The driven trajectory is taken as ground truth for the evaluation. The results show that on highways the quality functions perform nearly equal. However on country roads there are differences. The huge error reduction achieved by the range estimation is pointed out. At the end the quality function performing best is determined.

An automatic parking system relies on precise estimation of parking space geometry. This paper proposes the use of a hierarchical three dimensional occupancy grid for the detection of parking spaces. The occupancy grid covers the environment representation of the static world. A hierarchical design allows dynamic selection of the level of detail. Applying a three-dimensional grid provides the additional benefit of supporting a variety of other functions including height estimation using a single environment representation type. The presented approach derives the distance to obstacles and walls and thus is able to represent the free space that forms parking spaces. In a second step, the dimensions of the parking space are calculated. For evaluation, real parking spaces are detected and estimated using short range radar sensors. The calculated dimensions are compared to the ground truth.

This paper presents a novel and robust method to determine the outline contour of vehicles in 3D-LIDAR (Light Detection and Ranging) measurements. To calculate the outline contour, a vehicle is described by its geometrical properties. These properties are used as constraints to fit a surface to unordered, scattered and error-contaminated 3D measurements. The surface can be used to calculate a corresponding 2D outline contour. The algorithm is tested with two different laser scanners. One scanner has 64, the other only 4 layers.

In an automotive pre-crash application, it is vital to quickly and accurately estimate the position and velocity of objects in the frontal area of the vehicle. To improve such estimations, several radar sensors are fused to detect objects. Due to their different performance characteristics, their measurements can arrive at the pre-crash processing unit out-of-sequence. This work presents several techniques to integrate measurements into a tracking algorithm that arrive with such an out-of-sequence measurement (OOSM) scenario. A comprehensive complexity analysis of the algorithms is also presented. Most importantly, the algorithms are run on a test vehicle during real crash scenarios. The algorithms' performance is evaluated against reference data from a highly accurate laser scanner. It is shown that using advanced OOSM algorithms in pre-crash systems significantly increases performance and reduces computational cost compared to previous approaches.

Modern driver assistance and safety systems need a reliable and precise description of the environment. Fusing the measurement data of two or more sensors can improve the performance of the perception system. A generic fusion system which is independent of the attached sensors could be reused in multiple fusion systems and sensor combinations. This could be very helpful because sensor data fusion is a demanding and complex task. In this contribution, we present the algorithmic basics for a generic fusion system, detailed ways on how to model sensor specific properties and which benefits we can achieve by using these models.

In this paper, a generic approach for three-dimensional environment representation is presented. Scans from range finders are accumulated into a three-dimensional occupancy grid. A probabilistic measurement model is used to represent measurement uncertainties. Free regions are modelled as well and contribute to a precise representation of the environment. In order to acquire a feasible three-dimensional grid, a hierarchical data structure is proposed. As a positive result, the level of detail and respectively the grid resolution is controllable by the application or by the content. As an example application, it will be shown how height information can be derived from a sensor with one horizontal scan plane only.

In this paper, we present a vehicle safety application based on data gathered by a laser scanner and two short-range radars that recognize unavoidable collisions with stationary objects before they take place to trigger restraint systems. Two different software modules that perform the processing of raw data and deliver a description of the vehicle's environment are compared. A comprehensive experimental evaluation based on relevant crash and noncrash scenarios is presented.