Inaccurate freespace detection remains a significant challenge to the safety of autonomous driving. However, we observe that current multisource fusion approaches rely on converting LiDAR point clouds into depth maps, often lose crucial 3D geometric cues. This compromises the spatial consistency of predictions, especially in complex urban scenes. To address this limitation, we propose LiDAR-DHMT (LiDAR-Adaptive Dual-branch Hierarchical Mask Transformer), a novel framework designed for spatial-consistent freespace detection and semantic segmentation. Our key innovation lies in the introduction of a 3D Relative Position Bias module, which effectively captures LiDAR’s inherent spatial priors. This is coupled with a Dynamic Bias Attention mechanism that adaptively incorporates the 3D positional cues into the Transformer’s attention computation, enhancing spatial coherence. Additionally, we employ a Mask Interaction module and a global-local fusion strategy to jointly model contextual semantics and fine-grained structural details. Extensive experiments conducted on the KITTI Road, KITTI-360, Cityscapes datasets demonstrate that LiDAR-DHMT consistently outperforms existing state-of-the-art methods, achieving a competitive 97.59% F1 score in freespace detection and 69.45% and 84.4% mIoU in semantic segmentation. Our findings suggest that LiDAR-DHMT offers a practical solution for deploying robust freespace perception in complex urban driving environments.

Semantic scene completion (SSC) aims to reconstruct a complete 3D scene from a sparse point cloud while simultaneously predicting semantic labels. However, existing SSC methods still struggle with two major types of completion errors. First, when the geometric completion branch lacks an immediate understanding of semantic transitions at object boundaries, it tends to prioritize surface continuity or smoothness, leading to erroneous overfilling. Second, in severely occluded regions, insufficient understanding of scene context and object semantics often leads to implausible or incomplete reconstructions. To address these challenges, we propose Efficient-SSC , an efficient framework composed of three complementary modules. First, we introduce the Structural Semantic Joint ( SSJ ) module, which implicitly fuses structural cues and semantic features through a Structural Semantic Attention mechanism, thereby preserving geometric consistency and reducing superfluous filling in regions with clear boundaries. Second, the Hierarchical Region Structure Extraction ( HRSE ) module captures scene layout and structural relationships through multi-scale graph convolution to provide semantic priors for scene completion, thereby improving completion integrity. Finally, we develop the Awareness Offset Estimation ( AOE ) module, which jointly refines geometric structure and semantic information through a heterogeneous densification strategy, enabling predictive refinement and improving the fidelity of fine-grained scene details. Our network adopts a single-branch architecture for semantic scene completion. The proposed approach is computationally efficient and achieves competitive performance on widely used benchmark datasets, including SSC-PC and NYUCAD-PC. For example, on the SSC-PC dataset, compared with previous single-modality methods, it improves mIoU and mAcc by 2.54% and 1.82% , respectively, while reducing the parameter count by 28% .

Roof wireframe reconstruction has shown great success in 3D building reconstruction due to its lightweight nature and straightforward representation. However, previous methods consider all roof points, which result in edge redundancy and omissions. In this paper, we propose a novel and streamlined Edge-guided Geometric wireframe reconstruction framework, named EDGE. We find that points distributed along the roof edges make a significant contribution to the precise geometric structure of wireframe. Therefore, we design an edge point extractor (EPE) to capture the spatial relationship between points and edges, filtering out internal plane points. Moreover, we discover that the previous edge detectors rely solely on corner points, leading to error accumulation. To address this, we present the Hybrid Edge Detector (HED) feeding corner points with edge contextual features, which not only enhances edge completeness but also mitigates edge redundancy. Comprehensive experiments demonstrate EDGE outperforms existing wireframe reconstruction methods with 0.86 Corner F1-score and 0.71 Edge F1-score on Building3D dataset, striking the significant improvement of accuracy between corner and edge. Notably, our EDGE achieves a significant improvement of over 11% in Edge Recall, demonstrating the effectiveness and robustness of the proposed method.

Weakly supervised point cloud semantic segmentation (WSPCSS) has gained attention for reducing reliance on densely annotated data. However, inefficiencies in utilizing sparse annotations hinder comprehensive understanding of complex scenes. Inspired by masked autoencoder (MAE) techniques in image processing, researchers have adapted these methods to WSPCSS. Yet, current 3D masking strategies often fail to capture intricate geometric properties, resulting in generated to-be-filled content that inaccurately represents the underlying 3D scene structure. To address these limitations, this study proposes a novel stereo-wise masking strategy, which extends 2D plane masking into 3D space to generate coherent and semantically rich masked regions with contextual relevance. These regions serve as high-quality learning targets, enabling the model to better comprehend complex point cloud structures. Experimental results demonstrate that, at a 0.01 % annotation density, the proposed method achieves improvements in mIoU by 1.9 % and 4.72 % on the indoor datasets S3DIS and ScanNet V2, respectively, compared to previous methods. Furthermore, at a 0.1% annotation density on the forest dataset For-Instance, the method exhibits a 0.29 % improvement. These results substantiate the effectiveness and stability of the stereo-wise masking strategy.

RGB-D has gradually become a crucial data source for understanding complex scenes in assisted driving. However, existing studies have paid insufficient attention to the intrinsic spatial properties of depth maps. This oversight significantly impacts the attention representation, leading to prediction errors caused by attention shift issues. To this end, we propose a novel learnable Depth interaction Pyramid Transformer (DiPFormer) to explore the effectiveness of depth. Firstly, we introduce Depth Spatial-Aware Optimization (Depth SAO) as offset to represent real-world spatial relationships. Secondly, the similarity in the feature space of RGB-D is learned by Depth Linear Cross-Attention (Depth LCA) to clarify spatial differences at the pixel level. Finally, an MLP Decoder is utilized to effectively fuse multi-scale features for meeting real-time requirements. Comprehensive experiments demonstrate that the proposed DiPFormer significantly addresses the issue of attention misalignment in both road detection (+7.5%) and semantic segmentation (+4.9% / +1.5%) tasks. DiPFormer achieves state-of-the-art performance on the KITTI (97.57% F-score on KITTI road and 68.74% mIoU on KITTI-360) and Cityscapes (83.4% mIoU) datasets.

Open-Vocabulary semantic segmentation (OVSS) and domain generalization in semantic segmentation (DGSS) highlight a subtle complementarity that motivates Open-Vocabulary Domain-Generalized Semantic Segmentation (OV-DGSS). OV-DGSS aims to generate pixel-level masks for unseen categories while maintaining robustness across unseen domains, a critical capability for real-world scenarios such as autonomous driving in adverse conditions. We introduce Vireo, a novel single-stage framework for OV-DGSS that unifies the strengths of OVSS and DGSS for the first time. Vireo builds upon the frozen Visual Foundation Models (VFMs) and incorporates scene geometry via Depth VFMs to extract domain-invariant structural features. To bridge the gap between visual and textual modalities under domain shift, we propose three key components: (1) GeoText Query, which align geometric features with language cues and progressively refine VFM encoder representations; (2) Coarse Mask Prior Embedding (CMPE) for enhancing gradient flow for faster convergence and stronger textual influence; and (3) the Domain-Open-Vocabulary Vector Embedding Head (DOV-VEH), which fuses refined structural and semantic features for robust prediction. Comprehensive evaluation on these components demonstrates the effectiveness of our designs. Our proposed Vireo achieves the state-of-the-art performance and surpasses existing methods by a large margin in both domain generalization and open-vocabulary recognition, offering a unified and scalable solution for robust visual understanding in diverse and dynamic environments. Code is available at https://github.com/SY-Ch/Vireo.

ABSTRACT Remote sensing image segmentation is crucial for applications ranging from urban planning to environmental monitoring. However, traditional approaches struggle with the unique challenges of aerial imagery, including complex boundary delineation and intricate spatial relationships. To address these limitations, we introduce the semantic uncertainty‐aware segmentation (SUAS) method, an innovative plug‐and‐play solution designed specifically for remote sensing image analysis. SUAS builds upon the rotated multi‐scale interaction network (RMSIN) architecture and introduces the prompt refinement and uncertainty adjustment module (PRUAM). This novel component transforms original textual prompts into semantic uncertainty‐aware descriptions, particularly focusing on the ambiguous boundaries prevalent in remote sensing imagery. By incorporating semantic uncertainty, SUAS directly tackles the inherent complexities in boundary delineation, enabling more refined segmentations. Experimental results demonstrate SUAS's effectiveness, showing improvements over existing methods across multiple metrics. SUAS achieves consistent enhancements in mean intersection‐over‐union (mIoU) and precision at various thresholds, with notable performance in handling objects with irregular and complex boundaries—a persistent challenge in aerial imagery analysis. The results indicate that SUAS's plug‐and‐play design, which leverages semantic uncertainty to guide the segmentation task, contributes to improved boundary delineation accuracy in remote sensing image analysis.

Semantic segmentation of large-scale point clouds is an indispensable component of outdoor scene perception, providing essential 3-D semantic insights for applications in scene reconstruction, urban planning, autonomous driving, and more. However, the discriminative capability of point clouds features declines with increasing distance from the sensor, causing current methods to usually perform poorly in segmenting distant objects. To overcome this challenge and improve the differentiation between classes with similar geometric features, we propose the cross-modal semantic focus network (CSFNet). Firstly, we design a multiscale feature dynamic fusion (MDF) module to leverage multiscale image features, thereby enriching the feature representation of point clouds with additional images color and texture information. Then, in order to extract the distinguishing features of distant and different categories of objects more efficiently, we propose a semantic focus module (SFM) that employs a multiclass contrastive learning strategy to enhance feature discrimination. Finally, we introduce cross-modal knowledge distillation (KD) to augment the model’s comprehension of point clouds. Extensive experiments conducted on the SemanticKITTI and nuScenes datasets demonstrate the effectiveness of our method. Notably, our method achieves superior segmentation accuracy across multiple classes at various distances compared to current methods.

Semantic perception in driving scenarios plays a crucial role in intelligent transportation systems. However, existing Transformer-based semantic segmentation methods often do not fully exploit their potential in understanding driving scene dynamically. These methods typically lack spatial reasoning, failing to effectively correlate image pixels with their spatial positions, leading to attention drift. To address this issue, we propose a novel architecture, the Hierarchical Spatial Perception Transformer (HSPFormer), which integrates monocular depth estimation and semantic segmentation into a unified framework for the first time. We introduce the Spatial Depth Perception Auxiliary Network (SDPNet), a framework for multiscale feature extraction and multilayer depth map prediction to establish hierarchical spatial coherence. Additionally, we design the Hierarchical Pyramid Transformer Network (HPTNet), which uses depth estimation as learnable position embeddings to form spatially correlated semantic representations and generate global contextual information. Experiments on benchmark datasets such as KITTI-360, Cityscapes, and NYU Depth V2, demonstrate that HSPFormer outperforms several state-of-the-art networks, and achieves promising performance with 66.82% top-1 mIoU on KITTI-360, 83.8% mIoU on Cityscapes, and 57.7% mIoU on NYU Depth V2, respectively. The code will be made publicly available athttps://github.com/SY-Ch/HSPFormer.

When the number of available training views is limited, NeRF and 3DGS will soon overfit the optimization and learn the wrong scene geometry. For this challenge, a common solution is to provide depth prior as supervision to correct scene geometry. In this work, we present Geometric Regularized 3D Gaussian Splatting (GeoRGS), a priors-independent method for improving novel view synthesis from sparse inputs. We analyze the problems of the density control strategy in 3DGS with sparse inputs, and find that correcting the erroneous Gaussian growth trend at the beginning of training is effective in mitigating overfitting. Based on this analysis, we propose two geometric regularization methods that do not require prior information. One is based on selecting seed patches of 3D Gaussian from the scene, which guides growth to form correct scene geometry, while the other focuses on regularizing depth similarity between object surfaces and edges. GeoRGS achieves state-of-the-art performance in novel view synthesis from sparse input on LLFF, Blender, RealEstate10K and MipNeRF360 datasets, while also demonstrating significantly faster training speeds and rendering efficiency compared to other baselines.

High-performance and real-time road detection plays an essential role in Advanced Driver Assistance Systems (ADAS) of intelligent transportation. However, existing approaches still suffer from ambiguous road contour in traffic environment because deep learning methods lack explicit constraints on road boundaries with similar textures and structures. To address the unsatisfactory boundaries, we propose an efficient architecture for urban road detection to refine road edges adaptively. First, we design a lightweight symmetrical data-fusion network to merge spatial responses into visual features. Second, we construct cross-layer attention transformation to aggregate non-local contextual information. Moreover, a progressive uncertainty analysis module eliminates indistinct road and obstacle edges. Finally, we introduce upgrade uncertainty loss and improved deep supervision to constrain margin error for multi-scale predictions. Results of experiments using three famous datasets confirm the superiority of our method (F1-measure of 96.91% in KITTI, 98.86% in Cityscapes, and 95.18% in R2D, processing speed of 0.02s) over previous approaches. We demonstrate that, to ensure the safety of autonomous driving, the Epurate-Net adaptively refines road contour to reach exquisite road margins. The source code will be available soon.

Stereo matching is a prominent research area in autonomous driving and computer vision. Despite significant progress made by learning-based methods, accurately predicting disparities in hazardous regions, which is crucial for ensuring safe vehicle operation, remains challenging. The limitations of methods based on Convolutional Neural Networks (CNNs) are most noticeable in textureless regions and repetitive patterns, leading to unreliable predictions. Furthermore, calculating disparities for boundaries and thin structures, where the disparity jump phenomenon is prominent remains difficult. To address these issues, we propose a lightweight stereo matching architecture that focuses on obtaining real-time and high-precision disparity maps in hazardous areas. We exploit an efficient global enhanced path to provide global representations in ill-posed regions, where CNN-based approaches often struggle. Second, our model integrates local and global features to generate more reliable cost volume. Finally, our innovative uncertainty-aware module refines disparity, making full use of high-frequency detailed information and uncertainty attention, effectively preserving complex structures. Comprehensive experimental studies on SceneFlow demonstrate our method outperforms state-of-the-art methods, achieving an End-Point Error (EPE) of 0.47 with only 3.60M parameters. The effectiveness of our method speed-accuracy trade-off is further confirmed by competitive results obtained from the KITTI 2012 and KITTI 2015 experiments. Code is available at: https://github.com/YJLCV/Guard-Net.

Abstract Stereo matching is a fundamental and long‐standing task in computer vision. Although learning‐based stereo matching algorithms have made remarkable progress, two major challenges still persist. Firstly, existing cost aggregation methods that use stacked three‐dimensional convolutions are complex, leading to heavy computation and memory costs. Secondly these methods continue to struggle with establishing reliable matches in weakly matchable such as that edges and thin structures. To overcome these limitations, we propose an accurate and efficient network called Attention‐guided Aggregation and Error‐aware Enhancement Network (AAEE‐Net). Our approach involves designing an Attention‐guided Aggregation Mechanism (AAM) based on simple image features. This mechanism uses attention weights generated from image features to guide cost aggregation with a more efficient and effective strategy. Additionally, we propose an Error‐aware Enhancement Module (EEM) that refines the raw disparity by combining high‐frequency information from the original image and warp error between the left and right views. EEM enables the network to learn error correction capabilities that produce excellent subtle details and sharp edges. The experimental results on the SceneFlow and KITTI benchmark datasets demonstrate that AAEE‐Net achieves state‐of‐the‐art performance with low inference time. The qualitative results show that AAEE‐Net significantly improves predictions, especially for thin structures.

3-D object detection is a vital part of outdoor scene perception. Learning the complete size and accurate positioning of objects from an incomplete point cloud spatial structure is essential to 3-D object detection. We propose a novel flexible 3-D heatmap auxiliary network (3-D HANet) for object detection. To obtain complete structure and location information from an incomplete point cloud structure, we propose a 3-D heatmap to reflect object information. Also, we design a plug-and-play auxiliary network based on 3-D heatmap, which improves the accuracy of the entire detection network without extra computation in the inference stage. We validate the 3-D HANet on the basis of three classic 3-D object detection networks: PointPillars, sparsely embedded convolutional detection (SECOND), and structure aware single-stage 3-D object detection from point cloud (SASSD). Experimental results show that our auxiliary network augments the feature extraction ability of the backbone network, which is manifested in that the predicted boxes and the ground-truth boxes are more suitable in size and more aligned in direction. Furthermore, we conducted verification experiments on the state-of-the-art (SOTA) detector, CasA, and made a further improvement on the official ranking of the KITTI dataset.

Instance segmentation based on 3D point cloud is a key step in scene understanding. It is widely used in indoor robot navigation, outdoor autonomous driving, and other fields. But research in this area is still in its infancy. Instance segmentation not only needs to predict the semantic label of each point but also the instance label of each point. Therefore, semantic segmentation can be considered the basis of instance segmentation to some extent. Based on this motivation, we designed a voxel-based branch based on convertible sparse convolution and residual optimization modules. We design a point-based branch so that the network can maintain high-resolution representation. Then the two branches are combined to optimize the semantic segmentation results. Breadth-first search (BFS) performs well in indoor point clouds and is simple to operate. Therefore, we use this clustering operation to group the points of the same instance to obtain the instance segmentation result. The proposed method was tested on the indoor dataset Scan-Net v2 and achieved relatively good instance segmentation precision.

Object detection in aerial images has received increasing attention during the last few years. Scale variation is one of the main challenges in large scene aerial images. Existing object detection pipelines usually detect objects of different scale objects at multiple scale layers. However, the conventional detection approaches with multi-scale density predictions could cause duplicate detections of the same object. In this paper, we proposed a multi-scale cascade guided detection framework (MCGNet) to address these issues by guiding the different scales in detector focus on different scale objects. In particular, we proposed a multi-scale cascade module to predict the different scale objects with an explicit constraint in the loss function. Experiments on benchmark DOTA show promising performance of MCGNet compared with other detectors. Code will be released at https://github.com/jason-su/MCGNET.