Neural implicit surface representation methods have recently shown impressive 3D reconstruction results. However, existing solutions struggle to reconstruct driving scenes due to their large size, highly complex nature and their limited visual observation overlap. Hence, to achieve accurate reconstructions, additional supervision data such as LiDAR, strong geometric priors, and long training times are required. To tackle such limitations, we present ViiNeuS, a new hybrid implicit surface learning method that efficiently initializes the signed distance field to reconstruct large driving scenes from 2D street view images. ViiNeuS’s hybrid architecture models two separate implicit fields: one representing the volumetric density of the scene, and another one representing the signed distance to the surface. To accurately reconstruct urban outdoor driving scenarios, we introduce a novel volume-rendering strategy that relies on self-supervised probabilistic density estimation to sample points near the surface and transition progressively from volumetric to surface representation. Our solution permits a proper and fast initialization of the signed distance field without relying on any geometric prior on the scene, compared to concurrent methods. By conducting extensive experiments on four outdoor driving datasets, we show that ViiNeuS can learn an accurate and detailed 3D surface representation of various urban scene while being two times faster to train compared to previous state-of-the-art solutions.

Neural Radiance Fields (NeRF) enable 3D scene reconstruction from 2D images and camera poses for Novel View Synthesis (NVS). Although NeRF can produce photorealistic results, it often suffers from overfitting to training views, leading to poor geometry reconstruction, especially in low-texture areas such as road surfaces in driving scenarios. This limitation restricts many important applications which require accurate geometry, such as extrapolated NVS, HD mapping, simulation and scene editing. To address this limitation, we propose a new method to improve NeRF’s 3D structure using only RGB images and semantic maps. Our approach introduces a novel plane regularization based on Singular Value Decomposition (SVD), that does not rely on any geometric prior. In addition, we leverage the Structural Similarity Index Measure (SSIM) in patch-based loss design to properly initialize the volumetric representation of NeRF. Quantitative and qualitative results show that our method outperforms popular regularization approaches in accurate geometry reconstruction for large-scale outdoor scenes and achieves comparable rendering quality to SOTA methods on the KITTI-360 NVS benchmark.

In rapidly-evolving domains such as autonomous driving, the use of multiple sensors with different modalities is crucial to ensure high operational precision and stability. To correctly exploit the provided information by each sensor in a single common frame, it is essential for these sensors to be accurately calibrated. In this paper, we leverage the ability of Neural Radiance Fields (NeRF) to represent different sensors modalities in a common volumetric representation to achieve robust and accurate spatio-temporal sensor calibration. By designing a partitioning approach based on the visible part of the scene for each sensor, we formulate the calibration problem using only the overlapping areas. This strategy results in a more robust and accurate calibration that is less prone to failure. We demonstrate that our approach works on outdoor urban scenes by validating it on multiple established driving datasets. Results show that our method is able to get better accuracy and robustness compared to existing methods.

Reliable multimodal sensor fusion algorithms require accurate spatiotemporal calibration. Recently, targetless calibration techniques based on implicit neural representations have proven to provide precise and robust results. Nevertheless, such methods are inherently slow to train given the high computational overhead caused by the large number of sampled points required for volume rendering. With the recent introduction of 3D Gaussian Splatting as a faster alternative to implicit representation methods, we propose to leverage this new rendering approach to achieve faster multi-sensor calibration. We introduce 3DGS-Calib, a new calibration method that relies on the speed and rendering accuracy of 3D Gaussian Splatting to achieve multimodal spatiotemporal calibration that is accurate, robust, and with a substantial speed-up compared to methods relying on implicit neural representations. We demonstrate the superiority of our proposal with experimental results on sequences from KITTI-360, a widely used driving dataset.

Beyond novel view synthesis, Neural Radiance Fields (NeRF) are useful for applications that interact with the real world. In this paper, we use them as an implicit map of a given scene and propose a camera relocalization algorithm tailored for this representation. The proposed method enables to compute in real-time the precise position of a device using a single RGB camera, during its navigation. In contrast with previous work, we do not rely on pose regression or photometric alignment but rather use dense local features obtained through volumetric rendering which are specialized on the scene with a self-supervised objective. As a result, our algorithm is more accurate than competitors, able to operate in dynamic outdoor environments with changing lightning conditions and can be readily integrated in any volumetric neural renderer.

With the recent advances in autonomous driving and the decreasing cost of LiDARs, the use of multimodal sensor systems is on the rise. However, in order to make use of the information provided by a variety of complimentary sensors, it is necessary to accurately calibrate them. We take advantage of recent advances in computer graphics and implicit volumetric scene representation to tackle the problem of multi-sensor spatial and temporal calibration. Thanks to a new formulation of the Neural Radiance Field (NeRF) optimization, we are able to jointly optimize calibration parameters along with scene representation based on radiometric and geometric measurements. Our method enables accurate and robust calibration from data captured in uncontrolled and unstructured urban environments, making our solution more scalable than existing calibration solutions. We demonstrate the accuracy and robustness of our method in urban scenes typically encountered in autonomous driving scenarios.

This paper deals with the design of a new robust adaptive controller for parallel manipulators based on sliding mode and modelbased adaptive control. More precisely, the proposed controller relies on continuous finite-time terminal sliding mode (TSM) control and the linear-in-the-parameters property of the inverse dynamics of the manipulator. The main motivation behind the proposed scheme is to improve the tracking performance of fast and accurate parallel manipulators while guaranteeing the closed-loop system's robustness. Based on the linear-in-the-parameters property of the inverse dynamics of the manipulator, an adaptive law is proposed in order to estimate in real-time the dynamic parameters of the manipulator. The proposed controller has the advantage of relying on the desired reference trajectories instead of measured ones which can improve its robustness and efficiency. To demonstrate the effectiveness of the proposed controller, real-time experiments are conducted on a four-degree-of-freedom parallel manipulator called Veloce.

In this paper, the recently developed ℒ1adaptive control strategy is experimentally validated for the first time on a parallel kinematic manipulator. The ℒ1adaptive controller is known for its decoupled estimation and control loops which enables fast adaptation while guaranteeing robustness of the closed-loop system. The control scheme is experimentally implemented on a 4-DOFs parallel kinematic manipulator. Based on the obtained experimental results, a comparative study shows that the proposed ℒ1adaptive controller outperforms the PD controller in terms of tracking performance thanks to the compensation of the nonlinearities in the adaptive controller.

In this paper, a new control scheme based on the desired compensation adaptive control strategy for mechanical manipulators is developed. In order to estimate the unknown parameters, the adaptation law is formulated based on the inverse dynamic model and the desired trajectories instead of the actual ones. To further improve the tracking performance and the disturbance rejection ability of the original controller, the static feedback gains are replaced by nonlinear varying ones. The computed control inputs are then projected using a kinematics based projector in order to remove the internal efforts in redundantly actuated parallel kinematic manipulators that may damage the mechanical structure of the manipulator. To demonstrate its effectiveness, the proposed controller is validated through real-time experiments on Dual-V; a 3-DOFs redundantly actuated parallel kinematic manipulator. The obtained results show that the proposed controller outperforms the original one in terms of tracking errors and energy consumption.

A novel adaptive controller based on the Robust Integral of the Sign of the Error (RISE) is proposed. The RISE feedback strategy yields semi-global asymptotic tracking despite the presence of unstructured additive disturbances provided some limited assumptions on the system. To achieve better tracking performance, the RISE controller is extended with a model-based adaptive feedforward term. The addition of the feedforward term compensates for the structured uncertainties yielding reduced tracking errors and reduced control effort. The proposed controller is experimentally implemented on a 3-DOFs redundantly actuated parallel manipulator. The computed control inputs are projected using a kinematics based projector in order to remove the internal efforts that may damage the mechanical structure of the manipulator. Experimental results show a better performance of the proposed adaptive controller compared to the basic RISE controller in terms of tracking accuracy and energy consumption.