Large vision-language models (VLMs) for autonomous driving (AD) are evolving beyond perception and cognition tasks toward motion planning. However, we identify two critical challenges in this direction: (1) VLMs tend to learn shortcuts by relying heavily on history input information, achieving seemingly strong planning results without genuinely understanding the visual inputs; and (2) the chain-of-thought (COT) reasoning processes are always misaligned with the motion planning outcomes, and how to effectively leverage the complex reasoning capability to enhance planning remains largely underexplored. In this paper, we start from a small-scale domain-specific VLM and propose Drive-R1, designed to bridge the scenario reasoning and motion planning for AD. Drive-R1 first undergoes the supervised finetuning on an elaborate dataset containing both long and short COT data. Drive-R1 is encouraged to reason step-by-step from visual input to final planning decisions. Subsequently, Drive-R1 is trained within a reinforcement learning framework that incentivizes the discovery of reasoning paths that are more informative for planning, guided by rewards based on predicted trajectories and meta actions. Experimental evaluations on the nuScenes and DriveLM-nuScenes benchmarks demonstrate that Drive-R1 achieves superior performance compared to existing state-of-the-art VLMs. We believe that Drive-R1 presents a promising direction for bridging reasoning and planning in AD, offering methodological insights for future research and applications.

Recently, Bird’s-Eye View (BEV) representation has received increasing attention in multi-view 3D object detection. However, training high-performance camera-based BEV 3D detectors typically requires cumbersome annotated training samples. In this challenging scenario, we are the pioneers in addressing the problem of semi-supervised BEV 3D object detection. To this end, we propose object-guided semi-supervised BEV 3D object detection (OSS3D), which trains the 3D detectors by using a small amount of labeled data and a large amount of unlabeled data to reduce the cost of data labeling. Firstly, our approach employs an object-guided Gaussian-like mask generation paired with multi-perspective geometric correspondence to emphasize foreground regions and mitigate background noise, ensuring accurate object alignment across different spaces. Additionally, this Gaussian-like mask is integral to our object-guided deep feature map supervision strategy. Secondly, in order to avoid the impact of unreliable pseudo-labels on student network training and improve the importance of object depth information in real space, we propose a self-training depth-based refinement algorithm to generate high-quality and stable pseudo-labels. Finally, our method can be easily inserted into a variety of BEV 3D detection networks. Several experiments demonstrate that the proposed method outperforms current semi-supervised approaches by 2.59% mAP and 2.94% NDS in the nuScenes dataset, achieving new state-of-the-art (SOTA) compared to various camera-based 3D detectors. Code will be available athttps://github.com/yangzhaojason/OSS3D

Existing benchmarks for Vision-Language Model (VLM) on autonomous driving (AD) primarily assess interpretability through open-form visual question answering (QA) within coarse-grained tasks, which remain insufficient to assess capabilities in complex driving scenarios. To this end, we introduce $\textbf{VLADBench}$, a challenging and fine-grained dataset featuring close-form QAs that progress from static foundational knowledge and elements to advanced reasoning for dynamic on-road situations. The elaborate $\textbf{VLADBench}$ spans 5 key domains: Traffic Knowledge Understanding, General Element Recognition, Traffic Graph Generation, Target Attribute Comprehension, and Ego Decision-Making and Planning. These domains are further broken down into 11 secondary aspects and 29 tertiary tasks for a granular evaluation. A thorough assessment of general and domain-specific (DS) VLMs on this benchmark reveals both their strengths and critical limitations in AD contexts. To further exploit the cognitive and reasoning interactions among the 5 domains for AD understanding, we start from a small-scale VLM and train the DS models on individual domain datasets (collected from 1.4M DS QAs across public sources). The experimental results demonstrate that the proposed benchmark provides a crucial step toward a more comprehensive assessment of VLMs in AD, paving the way for the development of more cognitively sophisticated and reasoning-capable AD systems.

Recently, camera-based Bird's-Eye View (BEV) representation has gained significant traction in 3D object detection. However, training high-performance BEV 3D detectors typically requires a large number of annotated samples, which can be costly. Traditional semi-supervised methods for BEV 3D object detection face challenges including loss of rich depth information, inconsistent object representations across spaces, and unreliable pseudo label generation, leading to decreased accuracy and performance. Addressing this challenge, we pioneer the introduction of a semi-supervised BEV 3D object detection framework. Our approach leverages a small set of labeled data alongside a larger set of unlabeled data, significantly reducing annotation costs while maintaining robust detection performance. Firstly, we propose a depth-based self-refinement module to generate high-quality and stable pseudo labels, which can effectively regulate training with noisy labels. Secondly, we designed a denoising labels regression module that integrates denoising for both labeled and unlabeled data. Thirdly, in order to alleviate object inconsistency, we propose a consistent object-guided alignment method to ensure the consistency of objects in multi-spaces. Finally, our method can be easily plugged into various BEV 3D detection networks. Extensive experiments show that the proposed method achieves a new state-of-the-art compared to various camera-based 3D detectors tested on multiple public autonomous driving datasets.

Monocular online map segmentation is of great significance to mapless autonomous driving, and the core step is the View Transformation Module (VTM), which is used to transfer feature from the image perspective to the Bird-Eye-View (BEV). Most existing methods directly draw from the field of 3D object perception, either projecting 2D features into 3D space based on depth estimation, or projecting 3D coordinates into 2D images to query corresponding features, while ignoring the geometry and semantics from the ground surface. In this paper, we proposed a ground aware forward-backward view transformation module. The forward projection is used to generate the initial sparse BEV features and the geometric and semantic prior information of the ground surface. The backward module refines the BEV features based on the geometric and semantic priors, thereby improving the accuracy of map segmentation. In addition, the data partitioning of most previous related works has the problem of data leakage, so we repartitioned and experimented on the nuScense data set to conduct a fair evaluation. Experimental results demonstrate that our method achieves the highest accuracy on the test set. Code will be released at https://github.com/Brickzhuantou/MonoBEVseg.

LiDAR-Camera online calibration is of great significance for building a stable autonomous driving perception system. For online calibration, a key challenge lies in constructing a unified and robust representation between multi-modal sensor data. Most methods extract features manually or implicitly with an end-to-end deep learning method. The former suffers poor robustness, while the latter has poor interpretability. In this paper, we propose CalibDepth, which uses depth maps as the unified representation for image and LiDAR point cloud. CalibDepth introduces a sub-network for monocular depth estimation to assist online calibration tasks. To further improve the performance, we regard online calibration as a sequence prediction problem, and introduce global and local losses to optimize the calibration results. CalibDepth shows excellent performance in different experimental setups. Code is open-sourced at https://github.com/Brickzhuantou/CalibDepth.