3D synthetic-to-real unsupervised domain adaptive seg-mentation is crucial to annotating new domains. Self-training is a competitive approach for this task, but its performance is limited by different sensor sampling patterns (i.e., variations in point density) and incomplete training strate-gies. In this work, we propose a density-guided translator (DGT), which translates point density between domains, and integrates it into a two-stage self-training pipeline named DGT-ST. First, in contrast to existing works that simulta-neously conduct data generation and feature/output align-ment within unstable adversarial training, we employ the non-learnable DGT to bridge the domain gap at the in-put level. Second, to provide a well-initialized model for self-training, we propose a category-level adversarial net-work in stage one that utilizes the prototype to prevent neg-ative transfer. Finally, by leveraging the designs above, a domain-mixed self-training method with source-aware consistency loss is proposed in stage two to narrow the domain gap further. Experiments on two synthetic-to-real segmentation tasks (SynLiDAR → semanticKITTI and SynL- iDAR → semanticPOSS) demonstrate that DGT-ST outper-forms state-of-the-art methods, achieving 9.4% and 4.3% mIoU improvements, respectively. Code is available at https://github.com/yuan-zm/DGT-ST.

Compared to fully supervised 3D large-scale point cloud segmentation methods, which necessitate extensive manual point-wise annotations, weakly supervised segmentation has emerged as a popular approach for significantly reducing labeling costs while maintaining effectiveness. However, the existing methods have exhibited inferior segmentation performance and unsatisfactory generalization capabilities in some scenarios with unique structures (e.g., building facades). In this paper, we propose an effective and generalized weakly supervised semantic segmentation framework, called multi-stage scene-level constraints (MSC), to solve the above problem. To address the issue regarding inadequate labeled data, we use pseudo-labels for unlabeled data and propose an uncertainty-guided adaptive reweighting strategy to reduce the negative impact of erroneous pseudo-labeled data on the model learning process. To address the class imbalance issue, we employ multi-stage scene-level constraints (i.e., encoder, decoder, and classifier stages) to treat each class equally and improve perception ability of the model for each class. Evaluations conducted on multiple large-scale point cloud datasets collected in different scenarios, including building facades, indoor scenes, outdoor scenes, and UAV scenes, show that our MSC achieves a large gain over the existing weakly supervised methods and even surpasses some fully supervised methods.

Semantic segmentation of building facade point clouds has diverse applications. The development of semantic segmentation methods is inextricably linked to datasets. The available building facade datasets suffer from a lack of abundant semantic categories and data completeness. To compensate for these shortcomings, we propose a new building facade dataset characterized by various categories and relatively complete 3-D building facades. In addition, most existing methods focus on fully supervised learning, which relies on manually labeling large-scale point cloud data and results in high time and labor costs. In this article, we propose an effective weakly supervised building facade segmentation approach, called spatial adaptive fusion consistency contrastive constraint (SAF-C3), to solve the above problem. We first design a multirandom point cloud augmentor as an auxiliary supervision branch to enhance the learning ability of the original network branch. Then, we present a spatial adaptive fusion (SAF) module to extract discriminative features for building facade point clouds. Finally, we propose a spatial consistency contrastive constraint to explore the contrastive property in feature space and to ensure the predictive consistency among the augmentation and original branches. The proposed method achieves a significant performance improvement against the state-of-the-art methods on two building facade point cloud datasets through extensive experiments. In particular, the performance of SAF-C3 with 1% labels significantly surpasses the baseline network with 100% labels.

Unsupervised domain adaptation (UDA) segmentation aims to leverage labeled source data to make accurate predictions on unlabeled target data. The key is to make the segmentation network learn domain-invariant representations. In this work, we propose a prototype-guided multitask adversarial network (PMAN) to achieve this. First, we propose an intensity-aware segmentation network (IAS-Net) that leverages the private intensity information of target data to substantially facilitate feature learning of the target domain. Second, the category-level cross-domain feature alignment strategy is introduced to flee the side effects of global feature alignment. It employs the prototype (class centroid) and includes two essential operations: 1) build an auxiliary nonparametric classifier to evaluate the semantic alignment degree of each point based on the prediction consistency between the main and auxiliary classifiers and 2) introduce two class-conditional point-to-prototype learning objectives for better alignment. One is to explicitly perform category-level feature alignment in a progressive manner, and the other aims to shape the source feature representation to be discriminative. Extensive experiments reveal that our PMAN outperforms state-of-the-art results on two benchmark datasets.

Unsupervised domain adaptation (UDA) is a low-cost way to deal with the lack of annotations in a new domain. For outdoor point clouds in urban transportation scenes, the mismatch of sampling patterns and the transferability difference between classes make cross-domain segmentation extremely difficult. To overcome these challenges, we propose a category-level adversarial framework. Firstly, we propose a multi-scale domain conditioned block that facilitates to extract the critical low-level domain-dependent knowledge and reduce the domain gap caused by distinct LiDAR sampling patterns. Secondly, we make full use of multiple representation forms (i.e., point-based sets and voxel-based cells) and utilize the prediction consistency between the two forms to measure how well each point is semantically aligned. The model then focuses on the poorly-aligned points without affecting the well-aligned points. Experimental results on three autonomous driving point cloud datasets show that the proposed method outperforms existing methods by a large margin, especially on the low-beam to high-beam cross-domain segmentation task.

Automatic building facade point cloud semantic segmentation is an important step in 3-D urban building reconstruction. How to correctly segment the components (e.g., windows, walls, and columns) from the building facade is still a challenging task. According to the characteristics of building facade point clouds, we introduce local fusion attention network (LFA-Net), an efficient neural network that learns LFA features from building facade point clouds, for better capturing the local neighborhood structure information of each point. The core of LFA-Net is the LFA module, which consists of three neural units: local graph attention (LGA), local aggregation attention (LAA), and fusion attention (FA). The LFA-Net is the standard encoder-decoder architecture. Experiments demonstrate that our LFA-Net outperforms the state-of-the-art methods on the large-scale building facade point cloud dataset.

Successful remote sensing image registration is an important step for many remote sensing applications. The scale-invariant feature transform (SIFT) is a well-known method for remote sensing image registration, with many variants of SIFT proposed. However, it only uses local low-level information, and loses much middle- or high-level information to register. Image features extracted by a convolutional neural network (CNN) have achieved the state-of-the-art performance for image classification and retrieval problems, and can provide much middle- and high-level information for remote sensing image registration. Hence, in this letter, we investigate how to calculate the CNN feature, and study the way to fuse SIFT and CNN features for remote sensing image registration. The experimental results demonstrate that the proposed method yields a better registration performance in terms of both the aligning accuracy and the number of correct correspondences.