Autonomous driving systems are fundamentally composed of sequential modular tasks, i.e., perception, prediction, and planning. For connected autonomous vehicles (CAVs), cooperative perception offers a promising solution to surpass their perception limitations, such as occlusion, by sharing sensing data with each other through wireless communication. Existing works typically prioritize sharing data from potential object-containing areas to maximize object detection accuracy under constrained communication resources. However, such detection-oriented approaches ignore a crucial fact that more accurate detection does not equal safer planning. Sharing large amounts of sensing data for detection accuracy can lead to communication resource wastage and performance degradation of subsequent driving tasks. To address this, we introduce Plan2comm, a communication-efficient cooperative perception framework via planning-oriented feature sharing, which shares only sensing data around planned trajectories to enable safer planning rather than mere detection accuracy. Specifically, a planning-oriented communication mechanism is designed to select and transmit the most valuable features from the perspective of the planning task. Moreover, an uncertainty-aware spatial-temporal feature fusion strategy is proposed to enhance high-quality information aggregation. Comprehensive experiments demonstrate that Plan2comm outperforms all other cooperative perception methods on motion prediction performance, and is more communication-efficient.

Accurate object detection with on-board LiDAR sensors is crucial for ensuring driving safety of Connected Automated Vehicles (CAVs), especially at accident black spots with more occlusions. Fortunately, road-side infrastructure equipped with traffic cameras is usually available at these places, offers an extensive field of view and encounters fewer occlusions, and thus can provide sustained assistance to CAVs to improve their object detection performance. However, vehicle-to-infrastructure (V2I) cooperative object detection is quite challenging due to modality heterogeneity, agent heterogeneity, and bandwidth limitations. To address these challenges, in this paper, we propose V2I-Coop, an accurate object detection approach with V2I cross-modality cooperation for CAVs to improve perception performance at accident black spots. In V2I-Coop, first, we extract bird-eye-view (BEV) features from both multi-view 2D images and 3D point clouds, which facilitates the feature fusion of different modalities. Next, the most valuable features from the images are adaptively selected according to available bandwidth and then transmitted to CAVs. Then, a cross-modality feature fusion algorithm is adopted at CAVs to mitigate the modality difference and improve the feature fusion efficiency. Finally, extensive experiments demonstrate that V2I-Coop significantly improves the 3D object detection performance of CAVs at accident black spots.

Cooperative perception is an emerging paradigm that expects to conquer the sensory limitations of individual vehicles by sharing sensor information with each other and significantly improve driving safety. However, achieving highly precise data sharing and low communication overhead remains a challenge for cooperative perception, especially when real-time communication is necessary in autonomous driving. As a result, it is essential to decrease the transmitted sensor data while maintaining the perception performance. For this purpose, we propose a communication-efficient raw-level cooperative perception system for connected autonomous vehicles (CAVs), which is able to significantly compress the raw sensor data each CAV shares with each other by only transmitting the most informative keypoints. Specifically, at the local level, a voxel-based instance-aware keypoints selection strategy is proposed to select the points that belong to regions of interest. To further supervise the local keypoints selection, we present a collaborative global-local learning strategy, enabling each vehicle to consider both the local scenario and the global context when selecting the transmitted data. Comprehensive evaluations indicate the superiority of the proposed system, which achieves more than 300× lower communication volume compared to the raw data, with a performance degradation of less than 1%.

Cooperative perception is an effective way for connected autonomous vehicles (CAVs) to surpass their sensing limitations, by sharing information like intermediate features extracted from images or point clouds with each other. To reduce bandwidth consumption, feature filtering is adopted by existing methods to share only the most valuable information. However, these methods assume that the features on the same channel across all spatial regions or those in the same spatial regions across all the channels are equally important. This assumption results in coarse-grained feature filtering, which greatly decreases the cooperative perception performance. To solve this problem, this paper proposes a fine-grained channel-spatial feature filtering scheme, named SwissCheese, for communication-efficient cooperative perception. The key idea of SwissCheese is to exploit the disparity in semantic information on features between different spatial regions on different channels. Specifically, a fine-grained collaborative attention module is developed to jointly learn fine-grained attention along the channel-spatial dimensions. Moreover, a dual-dimensional feature selection strategy that selects sparse features for transmission based on the current available bandwidth is designed to achieve optimal perception performance. Experiment results show that SwissCheese significantly reduces the transmission data size by 90% with a subtle loss in perception performance.

Decentralized federated learning (DFL) is a promising technology to implement distributed machine learning in Internet of Vehicles (IoV), which enables vehicles to share and aggregate models with their neighbors in a vehicle-to-vehicle (V2V) network. However, due to the high mobility of vehicles, model sharing via V2V links may fail as the topology of the V2V network is time-varying, which greatly reduces the efficiency of model aggregating and the speed of model training. To address this problem, in this paper, we propose a graph convolution network (GCN)-based topology design method, named G-DFL, to improve the training efficiency of DFL in IoV by properly selecting a subgraph of the underlay V2V network, which is referred to as overlay network, in each round of model sharing. First, by encoding the state of vehicles, we utilize a GCN to extract the features of V2V network topology to predict the effective V2V links for model sharing. In addition, to further reduce the delay of model training, we use Christofides' Algorithm to find the Hamiltonian circuit with the least delay as the overlay network. Simulation results validate that the proposed method significantly improves the model training performance in DFL compared with the other baseline methods.

Cooperative perception is an effective way for connected autonomous vehicles to extend sensing range, improve detection precision, and thus enhance perception ability by combining their own sensing information with that of other vehicles. The existing cooperation perception schemes share only raw-, feature-, or object-level data, thus lacking the flexibility to adapt to highly dynamic vehicular network conditions, which leads to either bandwidth saturation or bandwidth underutilization, degrading the detection precision in the long run. In this article, we propose ML-Cooper, a multilevel cooperative perception framework, to fully utilize the bandwidth and hence improve detection precision. The key idea of ML-Cooper is to divide each frame of sensing data of the sender vehicle into three parts, and the corresponding raw data, feature data, and object data are transmitted to and fused at the receiver vehicle. We also develop a soft actor–critic (SAC) deep reinforcement learning algorithm to adaptively adjust the proportion of the three parts according to the channel state information of the Vehicle-to-Vehicle (V2V) link. The experimental results on KITTI and our collected data sets on two real vehicles show that ML-Cooper can achieve the highest average detection precision compared to existing single-level cooperative perception schemes.