To facilitate the large-scale deployment of autonomous driving in real-world scenarios, developing low-cost and high-performance 3D object detection systems has become a critical technical challenge. Although high-beam LiDARs provide denser point cloud data, their prohibitive hardware cost and high power consumption limit their practicality. In contrast, low-beam LiDARs offer advantages in terms of affordability and energy efficiency, but often suffer from inadequate perception accuracy due to their sparser point cloud data. This paper focuses on the task of multimodal 3D object detection with low-beam LiDARs, and proposes a novel approach that integrates temporal and spatial representation learning to enhance detection accuracy under sparser sensor conditions. Specifically, our approach comprises: (1) a Temporal Feature Prediction Learning (TFPL) module, which predicts the current BEV representation based on a sequence of historical BEV features; (2) a Spatial Feature Observation Learning (SFOL) module, which aligns BEV (Bird's-Eye-View) features from high-beam and low-beam LiDAR to enforce the low-beam features to approximate high-beam representations; (3) an Uncertainty-Aware Fusion (UAF) strategy, which performs feature-wise weighting between the predicted and observed BEV features by leveraging channel-wise variances, effectively mitigating perturbations in the learned BEV representations. Extensive experiments on the KITTI and nuScenes 3D object detection datasets demonstrate that the proposed approach significantly improves detection performance under low-beam LiDAR configurations.

In the 3D object detection task of autonomous driving systems, LiDAR and camera are the most crucial sensors, and current methods primarily focus on fusion strategies for these two complementary modalities. However, in real-world driving scenarios, potential sensor failures pose critical risks, potentially undermining the effectiveness of fusion-based detection approaches. This work presents RoLiC, a robust LiDAR-camera fusion framework designed to handle three challenging deployment scenarios within a unified model: LiDAR failure, camera failure, and simultaneous LiDAR-camera failure. To mitigate cross-modal dependency and recover missing information, RoLiC introduces two cross-modality feature transformers (L2C and C2L) that bidirectionally complete features between modalities when partial data is available. To further enhance feature reliability, we design a Sparse Similarity Loss (SSL) that constrains feature learning within high-probability object regions. Moreover, RoLiC integrates a task-aware two-stage feature knowledge distillation strategy (MS1 and MS2), where MS1 captures cross-modality complementarities, and MS2 distills knowledge between the fused modality and complete modality settings. Extensive experiments on the nuScenes and KITTI benchmarks demonstrate that RoLiC consistently outperforms state-of-the-art methods across all sensor-failure conditions. Code is available at https://github.com/JLIN77/RoLiC.

LiDAR and camera are the most common used sensors to percept the road scenes in autonomous driving. Current methods tried to fuse the two complementary information to boost 3D object detection. However, there are still two burning problems for multi-modality 3D object detection. One is the detection problem for the objects with sparse point clouds. The other is the misalignment of different sensors caused by the fixed physical locations. Therefore, this paper argues that explicitly fusing information from the two modalities with the physical misalignment is suboptimal for multi-modality 3D object detection. This paper presents a novel virtual point generation network, VirPNet, to overcome the multi-modality fusion challenges. On one hand, it completes sparse point cloud objects from image source and improves the final detection accuracy. On the other hand, it directly detects 3D targets from raw point clouds to avoid the physical misalignment between LiDAR and camera sensors. Different from previous point cloud completion methods, VirPNet fully utilizes the geometric information of pixels and point clouds and simplifies 3D point cloud regression into a 2D distance regression problem through a virtual plane. Experimental results on KITTI 3D object detection dataset and nuScenes dataset demonstrate that VirPNet improves the detection accuracy with the help of the generated virtual points.