Infrastructure-assisted autonomous driving has emerged as a pivotal technology to overcome the challenges posed by occlusions and limited fields of view for individual vehicles. Vehicles can fuse perception information from the infrastructure with their own in real-time, thereby enhancing their perception ability. However, our real-world experiments demonstrate that such an approach could introduce artifacts such as ghost objects, resulting in unsafe and unreliable planning outcomes. Besides, the system integration complexity and communication overhead are typically considerable, posing challenges to practical deployment. Therefore, we propose VI-Planning, an innovative infrastructure-assisted system that effectively optimizes autonomous vehicle planning in real time. The core idea of VI-Planning is to leverage the scene-level future occupancy grid maps constructed by the infrastructure as future drivable area references to directly optimize planning outcomes of autonomous vehicles. Since VI-Planning operates only at the autonomous vehicle's final output stage, without modifying the vehicle's underlying system architecture, it can be plug-and-play for most autonomous driving systems, whether they are modular or end-to-end architectures. Moreover, VI-Planning employs a novel bitwise encoding mechanism to efficiently compress these maps, enabling practical transmission. We implement VI-Planning end-to-end on a real-world testbed. The results of closed-loop and open-loop experiments indicate that VI-Planning can achieve real-time planning optimization (62.54 ms on average) and 817 × data transmission efficiency compared to the state-of-the-art baseline. A video demo of VI-Planning on our real-world testbed is available at: https://youtu.be/DXl5BhDEvFQ.

We propose VI-Planning, an innovative system that leverages the scene-level future occupancy grid maps predicted by the infrastructure as future drivable area references to directly optimize the planning trajectories of autonomous vehicles in real time. Since VI-Planning operates only at the autonomous vehicle's final output stage, without modifying the underlying system architecture, it can be plug-and-play for most autonomous driving systems. Moreover, VI-Planning employs a novel bitwise encoding mechanism to efficiently compress these maps, enabling practical transmission. The experimental results demonstrate that VI-Planning can achieve real-time planning optimization with extremely low bandwidth consumption, significantly enhancing the driving safety of autonomous systems. The source code and video demonstration of VI-Planning are available on GitHub: https://github.com/YANG-Deep/VI-Planning.

We propose VI-Planning, an innovative system that leverages the scene-level future occupancy grid maps predicted by the infrastructure as future drivable area references to directly optimize the planning trajectories of autonomous vehicles in real time. Since VI-Planning operates only at the autonomous vehicle's final output stage, without modifying the underlying system architecture, it can be plug-and-play for most autonomous driving systems. Moreover, VI-Planning employs a novel bitwise encoding mechanism to efficiently compress these maps, enabling practical transmission. The experimental results demonstrate that VI-Planning can achieve real-time planning optimization with extremely low bandwidth consumption, significantly enhancing the driving safety of autonomous systems. The source code and video demonstration of VI-Planning are available on GitHub: https://github.com/YANG-Deep/VI-Planning.

The century-old development of the automotive industry has spawned one of the greatest Cyber-Physical Systems (CPSs) in the future-unmanned vehicles. The vehicle can obtain environmental information through different sensors, map it to the virtual coordinate system of the vehicle body to make decisions, and finally generate control instructions. However, a series of factors, such as complex road scenes, defective and irregular target sparse sampling, and large coding space, pose challenges to accurate, efficient, and stable perception results. To overcome the most challenging problem of dynamic target tracking, this paper designs a two-stage detection model based on non-uniform polar voxelization sampling of irregular 3D point cloud, which is used with local registration-based search to achieve efficient multi-target tracking. Non-uniform voxelization not only balances the spatial sampling and encoding efficiency of the point cloud for the backbone, but also adapts to the feature aggregation of the detection head, thereby achieving double acceleration. Finally, we tested our model on KITTI Tracking data. The comparison results show that the calculation speed of the final model is greatly improved and the tracking accuracy is competitive in all categories.

Object detection is the most critical and foundational sensing module for the autonomous movement platform. However, most of the existing deep learning solutions are based on GPU servers, which limits their actual deployment. We present an efficient multi-sensor fusion based object detection model that can be deployed on the off-the-shelf edge computing device for the vehicle platform. To achieve real-time target detection, the model eliminates a large number of invalid point clouds through ground filtering algorithm, and then adds texture information (fused from camera image) through point cloud coloring to enhance features. The proposed PV-EncoNet efficiently encodes both the spatial and texture features of each colored point through point-wise and voxel-wise encoding, and then predicts the position, heading and class of the objects. The final model can achieve about 17.92 and 24.25 Frame per Second (FPS) on two different edge computing platforms, and the detection accuracy is comparable with the state-of-the-art models on the KITTI public dataset (i.e., 88.54% for cars, 71.94% for pedestrians and 73.04% for cyclists). The robustness and generalization ability of the PV-EncoNet for the 3D colored point cloud detection task is also verified by deploying it on the local vehicle platform and testing it on real road conditions.