A novel learning-optimization-combined 4D radar odometry model, named DNOI-4DRO, is proposed in this paper. The proposed model seamlessly integrates traditional geometric optimization with end-to-end neural network training, leveraging an innovative differentiable neural-optimization iteration operator. In this framework, point-wise motion flow is first estimated using a neural network, followed by the construction of a cost function based on the relationship between point motion and pose in 3D space. The radar pose is then refined using Gauss-Newton updates. Additionally, we design a dual-stream 4D radar backbone that integrates multi-scale geometric features and clustering-based class-aware features to enhance the representation of sparse 4D radar point clouds. Extensive experiments on the VoD and Snail-Radar datasets demonstrate the superior performance of our model, which outperforms recent classical and learning-based approaches. Notably, our method even achieves results comparable to A-LOAM with mapping optimization using LiDAR point clouds as input.

Place recognition is essential for achieving closedloop or global positioning in autonomous vehicles and mobile robots. Despite recent advancements in place recognition using 2D cameras or 3D LiDAR, it remains to be seen how to use 4D radar for place recognition - an increasingly popular sensor for its robustness against adverse weather and lighting conditions. Compared to LiDAR point clouds, radar data are drastically sparser, noisier and in much lower resolution, which hampers their ability to effectively represent scenes, posing significant challenges for 4D radar-based place recognition. This work addresses these challenges by leveraging multimodal information from sequential 4D radar scans and effectively extracting and aggregating spatio-temporal features. Our approach follows a principled pipeline that comprises (1) dynamic points removal and ego-velocity estimation from velocity property, (2) bird's eye view (BEV) feature encoding on the refined point cloud, (3) feature alignment using BEV feature map motion trajectory calculated by ego-velocity, (4) multiscale spatio-temporal features of the aligned BEV feature maps are extracted and aggregated. Real-world experimental results validate the feasibility of the proposed method and demonstrate its robustness in handling dynamic environments. Source codes are available.

We introduce R2LDM, an innovative approach for generating dense and accurate 4D radar point clouds, guided by corresponding LiDAR point clouds. Instead of utilizing range images or bird’s eye view (BEV) images, we represent both LiDAR and 4D radar point clouds using voxel features, which more effectively capture 3D shape information. Subsequently, we propose the Latent Voxel Diffusion Model (LVDM), which performs the diffusion process in the latent space. Additionally, a novel Latent Point Cloud Reconstruction (LPCR) module is utilized to reconstruct point clouds from high-dimensional latent voxel features. As a result, R2LDM effectively generates LiDAR-like point clouds from paired raw radar data. We evaluate our approach on two different datasets, and the experimental results demonstrate that our model achieves 6- to 10-fold densification of radar point clouds, outperforming state-of-the-art baselines in 4D radar point cloud super-resolution. Furthermore, the enhanced radar point clouds generated by our method significantly improve downstream tasks, achieving up to 31.7% improvement in point cloud registration recall rate and 24.9% improvement in object detection accuracy.

The path tracking performance of global vehicle targets (GVTs) is crucial in closed-field testing for autonomous vehicles (AVs). However, the uncertain latency of 5G communication, when GVTs receive control commands from the cloud server, is likely to impact their path tracking performance adversely. To address this issue, we propose a cloud-based predictive path tracking control strategy considering the uncertain latency of 5G communication. Firstly, the 5G communication dataset from the closed test field is analyzed, and an LSTM-attention latency prediction model is established and validated. Subsequently, a nonlinear model predictive control (NMPC) algorithm that incorporates communication latency is formulated. Finally, real-world tests are conducted, and the results demonstrate that the proposed method significantly improves latency prediction accuracy and path tracking performance of GVTs.

Localization is crucial for Intelligent Transportation Systems (ITS) as it provides the precise position and orientation necessary for the operation of autonomous vehicles. Monocular vision-inertial odometry (VIO) has gained extensive application due to its high accuracy and cost-effectiveness. In this paper, we propose 4DRE-VIO, a novel 4D radar-enhanced monocular VIO system that achieves accurate and cost-effective pose estimation for autonomous vehicles in urban environments. By tightly integrating 4D radar measurements, we address the issues of scale ambiguity, scale drift, and dynamic object interference associated with monocular VIO. Specifically, we integrate 4D radar Doppler velocity with non-holonomic constraints (NHC) to provide a reliable velocity observation. Moreover, this velocity is directly used to construct a monocular scale observation, ensuring stable and accurate scale estimation. To mitigate the impact of dynamic objects on VIO, which is based on static environment assumptions, we introduce a 4D radar-enhanced visual semantic segmentation algorithm for dynamic object recognition. Additionally, we utilize a factor graph to tightly integrate measurements from the 4D radar, IMU, and monocular camera. The design of adjacent factors and optimal co-visible factors ensures the efficient use of visual information. An adaptive fusion strategy based on driving conditions maximizes the benefits of each sensor. We validate the effectiveness of the proposed method through extensive testing on a large dataset collected in urban environments, including open streets and residential districts. Compared to typical VIO and 4D radar-related odometry, our method demonstrates superior performance on our dataset and the public dataset.

In recent years, self-supervised methods for monocular depth estimation has rapidly become an significant branch of depth estimation task, especially for autonomous driving applications. Despite the high overall precision achieved, current methods still suffer from a) imprecise object-level depth inference and b) uncertain scale factor. The former problem would cause texture copy or provide inaccurate object boundary, and the latter would require current methods to have an additional sensor like LiDAR to provide depth ground-truth or stereo camera as additional training inputs, which makes them difficult to implement. In this work, we propose to address these two problems together by introducing DNet. Our contributions are twofold: a) a novel dense connected prediction (DCP) layer is proposed to provide better object-level depth estimation and b) specifically for autonomous driving scenarios, dense geometrical constrains (DGC) is introduced so that precise scale factor can be recovered without additional cost for autonomous vehicles. Extensive experiments have been conducted and, both DCP layer and DGC module are proved to be effectively solving the aforementioned problems respectively. Thanks to DCP layer, object boundary can now be better distinguished in the depth map and the depth is more continues on object level. It is also demonstrated that the performance of using DGC to perform scale recovery is comparable to that using ground-truth information, when the camera height is given and the ground point takes up more than 1.03% of the pixels. Code is available at https://github.com/TJ-IPLab/DNet.