The recent development of foundation models for monocular depth estimation such as Depth Anything paved the way to zero-shot monocular depth estimation. Since it returns an affine-invariant disparity map, the favored technique to recover the metric depth consists in fine-tuning the model. However, this stage is not straightforward, it can be costly and time-consuming because of the training and the creation of the dataset. The latter must contain images captured by the camera that will be used at test time and the corresponding ground truth. Moreover, the fine-tuning may also degrade the generalizing capacity of the original model. Instead, we propose in this paper a new method to rescale Depth Anything predictions using 3D points provided by sensors or techniques such as low-resolution LiDAR or structure-from-motion with poses given by an IMU. This approach avoids fine-tuning and preserves the generalizing power of the original depth estimation model while being robust to the noise of the sparse depth, of the camera-LiDAR calibration or of the depth model. Our experiments highlight enhancements relative to zero-shot monocular metric depth estimation methods, competitive results compared to fine-tuned approaches and a better robustness than depth completion approaches. Code available at github.com/ENSTA-U2IS-AI/depth-rescaling.

Autonomous navigation requires accurate and reliable localization. In urban environments, infrastructure such as buildings and bridges disrupts Global Navigation Satellite Systems (GNSS), which requires the implementation of robust perception systems combined with inertial navigation. Roadside poles like traffic signs or light poles can serve as stable landmarks for map-based localization. When georeferenced in high-definition vector maps, these features enable reliable localization through detection pipelines and data association methods. While lidar captures their 3D geometry, distinguishing mapped poles in raw point clouds remains challenging. To train pole detectors tailored to the specific map used, we propose an automatic annotation framework that integrates lidar data, a vector map, and offline semantic segmentation to generate precise labeled data. By combining annotated pole clusters from the map with semantic segmentation, annotation errors can be minimized. This enables the training of a map-specific classifier optimized to detect mapped poles while filtering out irrelevant structures. It eliminates the need for manual labeling and ensures adaptability to the map used for online localization. Thanks to real data acquired in real-world urban scenarios, we show that this approach enhances significantly localization accuracy.

Detecting road features is a key enabler for autonomous driving and localization. For instance, a reliable detection of poles which are widespread in road environments can improve localization. Modern deep learning-based perception systems need a significant amount of annotated data. Automatic annotation avoids time-consuming and costly manual annotation. Because automatic methods are prone to errors, managing annotation uncertainty is crucial to ensure a proper learning process. Fusing multiple annotation sources on the same dataset can be an efficient way to reduce the errors. This not only improves the quality of annotations, but also improves the learning of perception models. In this paper, we consider the fusion of three automatic annotation methods in images: feature projection from a high accuracy vector map combined with a lidar, image segmentation and lidar segmentation. Our experimental results demonstrate the significant benefits of multi-modal automatic annotation for pole detection through a comparative evaluation on manually annotated images. Finally, the resulting multi-modal fusion is used to fine-tune an object detection model for pole base detection using unlabeled data, showing overall improvements achieved by enhancing network specialization. The dataset is publicly available.

For autonomous navigation, high definition maps are a widely used source of information. Pole-like features encoded in HD maps such as traffic signs, traffic lights or street lights can be used as landmarks for localization. For this purpose, they first need to be detected by the vehicle using its embedded sensors. While geometric models can be used to process 3D point clouds retrieved by lidar sensors, modern image-based approaches rely on deep neural network and therefore heavily depend on annotated training data. In this paper, a 2D HD map is used to automatically annotate pole-like features in images. In the absence of height information, the map features are represented as pole bases at the ground level. We show how an additional lidar sensor can be used to filter out occluded features and refine the ground projection. We also demonstrate how an object detector can be trained to detect a pole base. To evaluate our methodology, it is first validated with data manually annotated from semantic segmentation and then compared to our own automatically generated annotated data recorded in the city of Compiègne, France.

Localization integrity consists in providing a real-time measure of the level of trust to be placed in the localization estimates as vehicles operate. It provides a means of knowing whether position estimates are usable for navigation purposes. This paper formalizes the integrity concept and its underlying principles. Vehicles operate in different navigation environments, and so multiple sensors are used to ensure the required performance. Different sources of error exist. They must be bounded according to the acceptable level of risk for the application. This paper presents a generic approach for addressing integrity. It combines measurement rejection (for measurements considered to be faults) and position error characterization. For this purpose, a multi-sensor data fusion with a Fault Detection and Exclusion algorithm is constituted using a bank of information filters. These filters allow detected faults to be isolated without any prior assumption regarding the number of simultaneous errors. In addition, external integrity is expressed as a Protection Level of the localization solution. It uses a Student’s$t$-distribution in order to bound the distribution of the position error applicable to small integrity risks after a learning step. The approach is tested on data acquired on public roads using an experimental vehicle equipped with off-the-shelf proprioceptive and exteroceptive sensors together with an HD map. The results obtained validate the proposed approach.

Although autonomous vehicle technology has evolved significantly in recent years, the navigation of self-driving vehicles in complex scenarios is still an open issue. One of the major challenges in these conditions is safe navigation on roads open to public traffic. The main issue is the interaction of the autonomous vehicle with regular traffic, as the behaviors and intentions of human-driven vehicles are hard to predict and understand. In this paper we propose a strategy to allow an autonomous vehicle to safely cross a multi-lane roundabout. Our approach uses a High-Definition (HD) map to predict at lane level the future situation, harnessing the concept of virtual instances of road users, which is a key concept in anticipating the situation in a roundabout that can be represented by a navigation graph with loops. This paper presents a methodology that uses intervals representing road occupancy by vehicles, with the road being widened to reflect uncertainties in localization. Our method safely avoids collisions and guarantees that no priority constraints are violated during the insertion maneuver. Moreover, the method does not provide an overly cautious insertion policy,i.e., an autonomous vehicle does not wait for a long time before the insertion. The performance of our strategy was evaluated using the SUMO simulation framework. To better evaluate the complexity of the simulation scenario, a highly interactive vehicle flow was generated using real dynamic traffic data from the INTERACTION dataset. We report real tests carried out with an experimental self-driving vehicle on a test circuit. Our results show that this approach is easy to integrate into an embedded system and that it allows roundabouts to be crossed with a high level of safety.

In dynamic localization problems, the observations used from exteroceptive sensors are usually obtained from a single measurement. However, there are cases where the current measurement is not sufficient to detect the referenced landmark or to get a sufficient level of accuracy. In this study, a point cloud accumulation strategy is used to improve the resolution of a LiDAR sensor along its sparse axis. In particular, we are interested in the detection of markings transverse to the road axis in order to improve the accuracy of localization of an autonomous vehicle when approaching intersections or roundabouts. We present a method that allows the construction of an accurate observation with an associated observation model based on a High-Definition (HD) map through an accumulation of scans as the vehicle moves, by compensating the vehicle motion. The parameters of the accumulator are studied in terms of detection and accuracy. The quality of the observations and their impact on the localization quality are analyzed using real experiments carried out with an experimental vehicle equipped with a low-cost GNSS receiver, dead-reckoning sensors and a ground truth system.

In the paper a fast and consistent method to manage uncertainties on detected traffic agents providing reliable results is presented. The information provided by a LiDAR-based object detector is combined with a high-definition map to identify the drivable space of the carriage way. Because the use of a HD map requires the use of a localization system, the uncertainty of the estimated pose shall be handled carefully. A novel approach taking into account the localization uncertainty in the perception task by direct propagation of it onto the LiDAR points is proposed. It is compared with a classical propagation that relies on linearized approximation. The good performances of this approach in terms of integrity are demonstrated by the use of real data acquired at the entrance of a roundabout being a particularly complex situation.

Autonomous vehicles navigation in complex scenarios is still an open issue. One of the major challenges is the safe navigation of autonomous vehicles on roads open to public traffic. Indeed, behaviors and intentions of human-driven vehicles are hard to predict and understand. In this paper, we propose a strategy to make an autonomous vehicle able to cross safely a roundabout. Our approach relies on High-Definition (HD) maps with lane level description which allows to predict the future situation thanks to the concept of virtual vehicles. This method handles safely collision avoidance and guarantees that no priority constraint is violated during the insertion maneuver without being overly cautious. The performance is evaluated with the SUMO simulation framework. An highly interactive vehicles flow has been generated using real data from the INTERACTION dataset. We also propose strategies to extend our algorithm to multi-lane roundabouts and report how these extensions behave in terms of safety and traffic flow.

For an autonomous vehicle, situation understanding is a key capability towards safe and comfortable decision-making and navigation. Information is in general provided by multiple sources. Prior information about the road topology and traffic laws can be given by a High Definition (HD) map while the perception system provides the description of the space and of road entities evolving in the vehicle surroundings. In complex situations such as those encountered in urban areas, the road user behaviors are governed by strong interactions with the others, and with the road network. In such situations, reliable situation understanding is therefore mandatory to avoid inappropriate decisions. Nevertheless, situation understanding is a complex task that requires access to a consistent and non-misleading representation of the vehicle surroundings. This paper proposes a formalism (an interaction lane grid) which allows to represent, with different levels of abstraction, the navigable and interacting spaces which must be considered for safe navigation. A top-down approach is chosen to assess and characterize the relevant information of the situation. On a high level of abstraction, the identification of the areas of interest where the vehicle should pay attention is depicted. On a lower level, it enables to characterize the spatial information in a unified representation and to infer additional information in occluded areas by reasoning with dynamic objects.

Localization is one of the main challenges to be addressed to develop autonomous vehicles able to perform complex maneuvers on roads opened to public traffic. Having an accurate dead-reckoning system is an essential step to reach this objective. This paper presents a dead-reckoning model for car-like vehicles that performs the data fusion of complementary and redundant sensors: wheel encoders, yaw rate gyro and steering wheel measurements. In order to get an accurate dead-reckoning system with a drift reduced to the minimum, the parameters have to be well calibrated and the procedure has to be simple and efficient. We present a method able to accurately calibrate the parameters without knowing the ground truth by using a Rauch-Tung-Striebel smoothing scheme which enables to obtain state estimates as close to the ground truth as possible. The smoothed estimates are then used within a optimization process to calibrate the model parameters. The method has been tested using data recorded from an experimental vehicle on public roads. The results show a significant diminution of the dead-reckoning drift compared to a commonly used calibration method. We evaluate finally the average distance a vehicle can navigate without exteroceptive sensors by using the proposed four-wheeled dead reckoning system.

Localization with high integrity is crucial for highly autonomous vehicles. This requires that the localization system send a warning to a client application when it should not be used. The concept of integrity was firstly developed for aviation applications and recently became an active research area for autonomous vehicles. GNSS information merged with dead reckoning sensors is not sufficient for lane level localization in all navigation environments. Map-aided localization with vision sensors is essential to provide redundant and complementary information. In this work, a multi-sensor data fusion method that takes advantage of a high definition (HD) map is presented and the integrity of the obtained solution is quantified. A Fault Detection and Exclusion (FDE) step is added to exclude the faulty measurements from the fusion procedure. A second step is to bound the estimation errors in the Along Track (AT) and Cross Track (CT) directions through Protection Levels (PL). For this step, the usual Gaussian distribution is replaced by a Student's distribution with an adapted degree of freedom chosen according to the navigation environment. The performance of the approach is evaluated with an experimental vehicle equipped with a camera able to detect up to four lane markings simultaneously.

This work describes a cooperative pose estimation solution where several vehicles can perceive each other and share a geometrical model of their shape via wireless communication. We describe two formulations of the cooperation. In one case, a vehicle estimates its global pose from the one of a neighbor vehicle by localizing it in its body frame. In the other case, a vehicle uses its own pose and its perception to help localizing another one. An iterative minimization approach is used to compute the relative pose between the two vehicles by using a LiDAR-based perception method and a shared polygonal geometric model of the vehicles. This study shows how to obtain an observation of the pose of one vehicle given the perception and the pose communicated by another one without any filtering to properly characterize the cooperative problem independently of any other sensor. Accuracy and consistency of the proposed approaches are evaluated on real data from on-road experiments. It is shown that this kind of strategy for cooperative pose estimation can be accurate. We also analyze the advantages and drawbacks of the two approaches on a simple case study.

Maps can greatly improve vehicle localization using perception sensors that detect features georeferenced in the map. This relies on two assumptions. Firstly, the detected features and the elements of the map have to be correctly associated. Secondly, the features of the map have to be accurately referenced. In this paper, solutions regarding these issues are presented. The case study of localization using a camera detecting road markings is considered. A Kalman smoothing process is used to obtain the best possible estimate of the trajectory that enables to evaluate the reliability of markings stored in the map. A likelihood maximization technique is used to best associate the observed markings to those referenced in the map. By using these two methods, map errors are detected after a first passage in an area and can be mitigated in later passes. Experimental results are reported to evaluate the performance of this approach. It is shown that mapping errors can be correctly handled.

For autonomous driving, lane level accurate localization is a necessity for complex driving maneuvers. Classical GNSS based methods are usually not accurate enough to have an unambiguous lane level localization. Having camera measurements such as lane marking detections along with high definition maps can enhance localization performance. In this paper, we are interested in high integrity localization, meaning being robust with low risk level. We propose a novel geometrical approach using horizontal protection levels on localization to propagate uncertainties and use lane markings to have an unambiguous map-matching. We demonstrate on real data that the algorithm can cope with high levels of noise on both localization and detection.

This paper proposes a set-membership based method for simultaneous localization and mapping. A box particle filter is exploited and improved to estimate robot states and feature positions. An interval constraint propagation is used to reduce box sizes, i.e., to decrease the uncertainty of the estimates. Buffers are also used to get q-satisfied results when empty estimates arise, on the one hand. On the other hand, historical data are used to improve the estimation through buffer contraction. Illustrations of the proposed method are given over simulations and experiments, with comparisons with a particle filter based method. The results show that the proposed method can reach the same simultaneous localization and mapping accuracy as a particle filter based method but with fewer particles. Moreover, this approach is comparatively more robust to system and measurement noises.

Despite the tremendous growth of research regarding fully autonomous vehicles in the past few years, many safety critical scenarios, such as the crossing of roundabouts, are still open issues. One of the main challenge is to deal with the lack of visibility in complex environments. Vehicle-to-vehicle and vehicle-to-infrastructure communications offer an appealing solution to handle these situations in a cooperative way. To avoid computing an ad hoc control strategy for every possible scenario, we propose in this paper to adapt the concept of virtual platooning to roundabout crossing. This idea allows a single platooning control law to handle complex scenarios such as intersection and roundabout crossings. This work combines the use of high definition maps and a curvilinear coordinates framework to deal with any kind of roundabouts. The proposed approach is not limited to communicating autonomous vehicles but can also be used with manually driven communicating vehicles or non-communicating vehicles with the help on the infrastructure. A formal proof of the correctness of this approach is given and simulations have been carried out with a high definition map of a real roundabout. We also introduce a novel graphical representation called safety diagram to study de performances of our approach.

Localization is a key problem for autonomous vehicle navigation. The use of high-definition maps and perception algorithms allows now to have lane-level accurate pose estimation in terms of cross-track and heading error. In this paper, we focus on the along-track localization of cooperative vehicles. We introduce a one-dimensional formulation of the localization problem by considering curvilinear coordinates. The covariance intersection filter is derived in one dimension leading to a minimum variable filter which allows multiple vehicles to cooperate while keeping consistent localization estimates. We show that the along-track localization error is directly dependent on the relative orientation between the trajectories followed by the cooperating vehicles. Experiments with two autonomous electric vehicles were conducted to evaluate the proposed approach.

Pedestrian detection is a well-studied problem. Even though many datasets contain challenging case studies, the performances of new methods are often only reported on cases of reasonable difficulty. In particular, the issue of small scale pedestrian detection is seldom considered. In this paper, we focus on the detection of small scale pedestrians, i.e., those that are at far distance from the camera. We show that classical features used for pedestrian detection are not well suited for our case of study. Instead, we propose a convolutional neural network based method to learn the features with an end-to-end approach. Experiments on the Caltech Pedestrian Detection Benchmark showed that we outperformed existing methods by more than 10% in terms of log-average miss rate.