Text-to-image diffusion model has inspired research into text-to-data synthesis without human intervention, where spatial attentions correlated with semantic entities in text prompts are primarily interpreted as pseudo-masks. However, these vannila attentions often deliver visual-linguistic discrepancies, in which the associations between image features and entity-level tokens are unstable and divergent, yielding inferior masks for realistic applications, especially in more practical open-vocabulary settings. To tackle this issue, we propose a novel text-guided self-driven generative paradigm, termed FreeGen, which addresses the discrepancies by recalibrating intrinsic visual-linguistic correlations and serves as a real-data-free method to automatically synthesize open-vocabulary pixel-level data for arbitrary entities. Specifically, we first learn an Attention Self-Rectification mechanism to reproject the inherent attention matrices to achieve robust semantic alignment, thereby obtaining class-discriminative masks. A Temporal Fluctuation Factor is present to assess mask quality based on its variation over uniform sampling timesteps, enabling the selection of reliable masks. These masks are then employed as self-supervised signals to support the learning of an Entity-level Grounding Decoder in a self-training manner, thus producing open-vocabulary segmentation results. Extensive experiments show that the existing segmenters trained on FreeGen narrow the performance gap with real data counterparts and remarkably outperform the state-of-the-art methods.

Accurate and reliable 3D object detection is crucial for autonomous driving, normally achieved using camera-only or camera-LiDAR fusion methods based on BEV (Bird’s Eye View) perspective. However, visual perception through cameras alone faces significant challenges, such as ambiguous depth estimation and poor performance in low-light, while camera-LiDAR fusion, though robust, are expensive on deployment and have limitations in dust and fog weather conditions. To address these issues, we propose RAFDet (Radar-Assisted Fusion Detection), a novel multi-modality fusion framework integrating camera and radar data for enhanced 3D detection. RAFDet utilizes radar RF (Radio Frequency) images to enrich spatial details, boosting fusion effects, while radar point clouds provide precise depth information to rectify visual depth predictions. In addition, the Edge-Awareness Feature Enhancement mechanism compensates for the sparsity of radar points, further refining depth estimation and detection accuracy. Our extensive experiments demonstrate the superiority of RAFDet over current methods, highlighting its potential for autonomous driving systems. Key results show significant improvements in depth prediction and overall performance metrics, validating the effectiveness of our approach. Our data collection and testing in real traffic scenarios also reflects the robustness, accuracy, and cost-effectiveness of RAFDet for 3D object detection in autonomous driving.

Modeling complicated interactions among the ego-vehicle, road agents, and map elements has been a crucial part for safety-critical autonomous driving. Previous work on end-to-end autonomous driving relies on the attention mechanism to handle heterogeneous interactions, which fails to capture geometric priors and is also computationally intensive. In this paper, we propose the Interaction Scene Graph (ISG) as a unified method to model the interactions among the ego-vehicle, road agents, and map elements. With the representation of the ISG, the driving agents aggregate essential information from the most influential elements, including the road agents with potential collisions and the map elements to follow. Since a mass of unnecessary interactions are omitted, the more efficient scene-graph-based framework is able to focus on indispensable connections and leads to better performance. We evaluate the proposed method for end-to-end autonomous driving on the nuScenes dataset. Compared with strong baselines, our method significantly outperforms in full-stack driving tasks.