Individual tree segmentation from terrestrial laser scanning (TLS) point clouds is essential for precise forest inventory, instance-level tree modeling, and the estimation of forest stock volume. However, current instance-level segmentation techniques encounter significant challenges in complex forest environments, particularly those characterized by dense understory vegetation and substantial crown overlap in natural forests. These complexities reduce segmentation accuracy and limit the generalizability of existing methods across diverse forest types. This paper presents a unified method for individual tree segmentation that integrates trunk localization with crown segmentation. The trunk localization uses normal vector features to eliminate non-trunk slice points, employs an enhanced DBSCAN algorithm for trunk slice separation, and refines trunk positions by fitting circular-like trunk slices using the Hough transform. This integrated approach ensures precise segmentation and optimization of final trunk positions. Subsequently, a graph-based optimization method is applied for crown segmentation. This method incorporates supervoxel technology, an optimal Euclidean distance metric between supervoxels, and a supervoxel similarity metric to construct an optimal undirected graph. Tree crown supervoxels are segmented by tracing the shortest path from the crown supervoxels to their corresponding tree roots. We validated the proposed method on eight sample plots representing various complexities and forest types. For tree trunk localization, the proposed method achieved an average Mean accuracy of 0.761, which is 27% higher than the best result among the three traditional methods. For crown segmentation, it achieved an average mIoU of 0.645, marking a 31% improvement over the best baseline performance. The source code for our individual tree segmentation method is available at https://github.com/TLS-tree/tree-segmentation.

Semantic segmentation of large-scale point cloud urban scenes faces significant challenges due to the complexity and diversity of object distribution. Transformer can effectively model the long-range dependencies in large-scale urban scenes, facilitating a comprehensive understanding of the overall structure and global context of urban scenes. Owing to the inherent strengths of the Transformer architecture, an increasing number of studies have applied it to the processing of large-scale point cloud data in complex scenes. But the excessive focus on global relationship inevitably leads to information redundancy, which can affect the model’s performance. To address this issue, we propose a novel Point Cloud Urban Scenes Semantic Segmentation network called PGFormer. This network consists of our proposed Group Representation Transformer (GRT) and KPConv, ensuring the enhancement of local key information while preserving global context. Specifically, in the proposed GRT block, we compute group-based and feature-based attention maps for Q and K across different groupings, and obtain the final attention output by integrating these with the global V, which has been pre-embedded with positional encoding derived from a triangular function. Our model is tested on two MLS (Paris-Lille-3D, Toronto3D) datasets and two ALS (Hessigheim 3D, ISPRS Vaihingen) datasets, and compared with a range of state-of-the-art (SOTA) methods, demonstrating the outstanding performance of our approach. Among various land cover segmentation tasks, our model achieves best or second-best results, particularly attaining the best OA (99.1%) and mF1-score (90.9%) on the Lille2 subset. Codes are available at https://github.com/Kange7/PGFormer.

Point cloud registration serves as a fundamental problem across multiple fields including computer vision, computer graphics, and remote sensing. While local feature descriptors (LFDs) have long been established as a cornerstone for point cloud registration and the LFD-based approach has been extensively studied, the field has witnessed significant advancements in recent years. Despite these developments, the research community lacks a systematic review to consolidate these contributions, leaving many researchers unaware of recent progress in LFD-based registration. To address this gap, we present a comprehensive review that critically examines both state-of-the-art and widely referenced methods across all subtasks of LFD-based registration. Our work provides: (1) an extensive survey of existing methodologies, (2) in-depth analysis of their respective strengths and limitations, (3) insightful observations and practical recommendations, and (4) a thorough summary of relevant applications and publicly available datasets. This systematic overview offers valuable guidance for researchers pursuing future investigations in this domain.

The application of deep learning-based semantic segmentation in remote sensing (RS) images has achieved considerable success. However, many supervised methods still heavily rely on a large amount of labeled data, requiring time-consuming and labor-intensive manual annotations. Besides, networks trained on labeled source domain data often perform poorly in inference tasks with target domain data due to the domain shift phenomenon. To address these challenges, we construct a novel end-to-end unsupervised domain adaptation (UDA) framework, named data augmentation and mobile threshold self-supervision (DA&MTSS), which integrates data augmentation with self-supervision. Specifically, we analyze the common factors that cause domain shifts in RS images and adopt different data augmentation techniques to attenuate the domain shift and enhance the generalization ability and robustness of the network in cross-domain inference. In the self-supervision phase, we design a new sample-based mobile threshold method to dynamically control the thresholds of both dominant and long-tail classes during training and generate stable pseudo-labels. Therefore, our method eliminates the need for additional training or expert knowledge and achieves the co-evolution of network parameters and pseudo-label quality in the training process. The results of comprehensive experiments on five tasks across the ISPRS Vaihingen, Potsdam, and LoveDA datasets demonstrate that this method consistently achieves higher mIoU scores, showcasing the performance advantage of DA&MTSS in UDA for the semantic segmentation of RS image.

The semantic segmentation network performance of large-scale outdoor point clouds is usually limited by the number of input point clouds. In the application of most methods, the point cloud is cut into small pieces as the training input, which will not only lead to heavy preprocessing burdens but also destroy the overall geometric structure of the scene. The transformer network has demonstrated remarkable advantages of attention mechanisms in focusing on crucial features and improving model performance. However, its training and inference on large-scale data are confined by computational complexity. To eliminate these challenges, the attention mechanisms are improved to enhance their performance in processing large-scale input data, while removing limitations imposed by computational complexity. Moreover, a novel local feature enhancement (LFE) module is developed to construct the LFE-Net, which can accurately and efficiently extract spatial and attribute features from local point clouds. In particular, an improved attention module, which is called simple offset attention (SOA), is adopted for local point cloud spatial feature learning. Compared with self-attention, SOA requires less memory and can better capture the fine-grained local features. Furthermore, to effectively avoid the destruction of object geometry and diminish the impact of sample imbalance, a training sample collection method based on the number of different classes is designed. To validate the effectiveness of this method, some experiments are conducted based on publicly accessible outdoor point cloud datasets. The results demonstrate that the LFE-Net can achieve substantial improvements compared with other cutting-edge network models.

The building contours, as one of the most important features for representing geometry, are widely used in various applications including urban modeling and reconstruction. Automatic extraction of high-fidelity compact contours from unstructured point clouds is rather challenging, and the existing methods are limited in generating global-aware and artifact-free building contours. Here, we approach contour extraction as a Bayesian inference problem. Two ideas are proposed to obtain building contour points and edges directly from unstructured point clouds. First, we construct a point-edge collaborative (PECo) Bayesian framework to couple the information of contour points and contour edges. The developed model fully takes local contour features, global edge structures, and global geometric priors into account. Second, given the Bayesian framework for building contouring, we leverage an expectation maximization (EM) algorithm to iteratively infer the contour edges in a maximum posteriori manner. The alternate EM iterations between point and edge domains progressively refine the local pointwise information to a global representation of contour edges. As a result, a synergistic effect between the point features and edge structures for global-aware building contouring is attained. Our approach outperforms the state-of-the-art methods in terms of geometric accuracy and structural compactness in modeling buildings with various complexities. Furthermore, the compactness of the contouring results can be more flexible and easily controlled.

Trees play a crucial role in promoting green, ecological, and low-carbon cities, with street trees being essential for urban roadways. Understanding the 3-D structure and biological characteristics of these trees requires accurate separation of wood and leaf. Mobile laser scanning (MLS) technology, known for its high efficiency and resolution, offers significant advantages. MLS data, however, often contain missing or overlapping areas due to occlusions and scanning geometry, complicating precise urban tree modeling. To address these challenges, this article introduces a coarse-to-fine approach for distinguishing wood from leaves in urban street trees. The proposed method begins with a hierarchical workflow that integrates the density-based spatial clustering of applications with noise (DBSCAN) algorithm to identify individual tree nodes. These nodes form the basis for constructing a graph structure for each tree. By leveraging prior knowledge of branch tilt angles, we enhance the shortest path algorithm, facilitating the extraction of features like shortest path frequency and length. This initial step completes a coarse differentiation between wood and leaves. To further refine accuracy, the identified wood and leaf points undergo analysis to extract multiscale geometric features. Integrating these features with the random forest (RF) algorithm results in a more precise separation of wood and leaf points. Our method demonstrates promising segmentation capabilities in MLS-captured roadside trees. Compared to four state-of-the-art methods for wood and leaf separation, our approach shows superior accuracy and efficiency, particularly in accurately identifying trunk points and minor branch points, as well as classifying the outer canopy layer.

Point cloud registration is crucial for various applications such as robotics and urban planning, yet it remains challenging due to issues such as noise, variable resolutions, and uncertainties in the initial pose. The normal distribution transform (NDT) algorithm has become a standard approach for registering point scans, but its application to large-scale scenes faces significant challenges due to inherent limitations. These include computational complexity, susceptibility to noise interference, limited adaptability to varying voxel resolutions, and substantial initial pose deviations. To address these issues, we propose the multiscale iterative NDT (MI-NDT) through a meticulously designed multiscale iterative optimization framework. The MI-NDT algorithm voxelizes the reference scan into multiscale voxels based on the optimal flatness ratio criterion. At each scale, the classic NDT algorithm computes the pose of the source scan. The pose obtained from coarse voxels serves as input for the next optimization step on finer voxels. This iterative refinement continues until the finest voxel level, where the resulting pose corresponds to the optimal translation parameters. The experimental results demonstrate the effectiveness of the MI-NDT algorithm in terms of registration accuracy and robustness. Comparative experiments with existing algorithms show significant improvements in accuracy, especially under varying voxel resolutions and noise levels. Moreover, the algorithm exhibits enhanced tolerance to initial pose deviations, enabling successful registration even with poor initial poses.

3D models of urban buildings have paramount importance to most digital urban applications. However, requirement of large storage and high computational cost for processing the geometric details of urban objects have been observed as a major limitation to existing 3D modeling approaches. This draws the need of lightweight modeling techniques requiring less computational storage to capture the details of the urban entities. Additionally these models should facilitate accelerated visualizations along with consuming lesser bandwidth for online applications. In this paper, we propose a lightweight urban modeling method using gradient structure tensors based feature point extraction to produce highly detailed lightweight 3D building models from LiDAR scans. Further, a mean cost-based edge collapse operation is proposed to preserve the feature points. The qualitative and quantitative analysis and comparative study of different building façade models shows the efficacy of our method in generating simplified models with a trade-off between model simplification and accuracy.

Graph Neural Networks have brought many breakthroughs in graph representation learning and boosted the state of the art in many data processing domains including point cloud learning. Graph structures provide the relative neighborhood information to unordered point clouds and help the exploration of topological structures for the understanding of complex scenes. Therefore, in this paper, we propose novel neural network architectures leveraging graph attention for point cloud-based classification and segmentation. The proposed architectures construct a connected graph from point cloud scans and employ a global aware attention module using global, local, and self-feature information for point cloud segmentation and classification. Various experiments on different datasets (ModelNet40 [1], ShapeNet [2], S3DIC [3], and Semantic3D [4]) show that our methods yield comparable results for classification, part segmentation, and semantic segmentation.

In this work, we present an attention based feature aggregation technique in graph neural networks (GNN) for detecting objects in LiDAR scan. We first employ a distance-aware downsampling scheme that not only enhances the algorithmic performance but also retains maximum geometric features of objects even if they lie far from the sensor. Our graph attention formulation uses novel neighborhood aggregation strategies through per node masked attention by combining local and self features. The experiments on KITTI dataset show that the proposed method yields comparable results for 3D object detection under GNN models category.

As many LiDAR point cloud processing steps, such as reconstruction, are often time- and memory-consuming, dividing LiDAR point clouds into subregions is common and necessary during preprocessing. However, the existing data dividing methods rely on tedious manual work or regular grids and result in oversegmentation around cutting lines. In this letter, we propose a new gap-based data dividing method for various LiDAR point clouds that can minimize the intersections between cutting lines and objects. The basic idea is to find a set of optimal paths that consist of gaps between objects as potential cutting lines. The experiments and comparisons in three data sets demonstrate that the proposed method is much better than the baseline method in terms visual inspection and cutting line quality.

Sliding-window-based low-rank matrix approximation (LRMA) is a technique widely used in hyperspectral images (HSIs) denoising or completion. However, the uncertainty quantification of the restored HSI has not been addressed to date. Accurate uncertainty quantification of the denoised HSI facilitates applications such as multisource or multiscale data fusion, data assimilation, and product uncertainty quantification since these applications require an accurate approach to describe the statistical distributions of the input data. Therefore, we propose a prior-free closed-form element-wise uncertainty quantification method for LRMA-based HSI restoration. Our closed-form algorithm overcomes the difficulty of handling uncertainty in HSI patch mixing caused by the sliding-window strategy used in the conventional LRMA process. The proposed approach only requires the uncertainty of the observed HSI and provides the uncertainty result relatively rapidly and with similar computational complexity as the LRMA technique. We conduct extensive experiments to validate the estimation accuracy of the proposed closed-form uncertainty approach. The method is robust to at least 10% random impulse noise at the cost of 10%–20% of additional processing time compared to the LRMA. The experiments indicate that the proposed closed-form uncertainty quantification method is more applicable to real-world applications than the baseline Monte Carlo test, which is computationally expensive.

The point cloud registration is important and necessary for the applications of changing detection, deformation monitoring, and so on, which is also challenging due to the vast clustered points, irregular, and complex structures of spatial objects, and quality effects of the labeled corresponding points. Aiming at this problem, we design an end-to-end 3-D graph deep learning framework of point cloud registration, which can simultaneously learn the detector (graph attention expression) and the descriptor (graph deep feature) for point cloud registration in a weakly supervised way, so that the learned detector and descriptor promote each other in the process of model optimization. Then, the detector is used to automatically extract the keypoints, and the descriptor describes the deep feature of each keypoint. In the framework, we innovatively propose a new module (named MLP_GCN), which fuses multilayer perceptron (MLP) and graph convolutional network (GCN). The MLP_GCN module is further integrated into the detector branch and descriptor branch to fully express the detector and descriptor of the point cloud. In the training process of the framework, we rotate and translate the point cloud randomly to form the training data in a weakly supervised way, which can save plenty of manually labeling time of corresponding points. In the experiments, our method can achieve better results of point cloud registration in comparison with other methods, which verifies the advantages of the proposed method.

Metro subway systems with underground tunnels form the backbone of urban transportations and therefore, accurate monitoring and maintenance of such subway systems are extremely necessary for a hassle-free daily commutation of billions of people. Though 3-D models of tunnels are widely used for the deformation monitoring of such subway tunnels, existing model-based tunnel monitoring systems rely on coarse geometric models and hence fail to capture complete tunnel health information. We present a two-stage algorithm to create high-fidelity geometric models of tunnel lining from Terrestrial Laser Scanning (TLS) point clouds. Tunnel geometry, defined at the detailed block entity level, is constructed through a data-driven block segmentation algorithm and a model-driven assembly technique. In our approach, the 3-D tunnel block segmentation problem has been translated into a bolt and lining joint recognition problem from 2-D images unfolded from the 3-D scans. The segmented 3-D blocks are matched with a set of predefined 3-D templates from a primitive library via a constraint total least squares matching method and the matched 3-D templates are assembled to create the final watertight tunnel model. The proposed tunnel modeling method has been comprehensively evaluated on Changzhou, Nanjing, and Wuhan tunnel data sets in terms of outliers, missing data, point density, topological representation, robustness, and geometric accuracy. The experiments on Nanjing and Changzhou metro tunnels show that the geometric model fitting incurs an error of only 7 mm, which is almost consistent with a mean density of 6 mm of these two data sets. Experimental results validate the advantages and potentials of the proposed tunnel modeling method.

Airborne light detection and ranging (LiDAR) data are widely applied in building reconstruction, with studies reporting success in typical buildings. However, the reconstruction of curved buildings remains an open research problem. To this end, we propose a new framework for curved building reconstruction via assembling and deforming geometric primitives. The input LiDAR point clouds are first converted into contours where individual buildings are identified. After recognizing geometric units (primitives) from building contours, we get initial models by matching the basic geometric primitives to these primitives. To polish assembly models, we employ a warping field for model refinements. Specifically, an embedded deformation (ED) graph is constructed via downsampling the initial model. Then, the point to model displacements is minimized by adjusting node parameters in the ED graph based on our objective function. The presented framework is validated on several highly curved buildings collected by various LiDAR in different cities. The experimental results, as well as accuracy comparison, demonstrate the advantage and effectiveness of our method. The new insight attributes to an efficient reconstruction manner. Moreover, we prove that the primitive-based framework significantly reduces the data storage to 10%-20% of classical mesh models.

Classification of airborne laser scanning (ALS) point clouds is needed in digital cities and 3-D modeling. To efficiently recognize objects in ALS point clouds, we propose a novel hierarchical aggregated deep feature representation method, which can adequately employ spatial association of multilevel structures and deep feature discrimination. In our method, a 3-D deep learning model is constructed to represent the discriminative feature of each point cluster in a hierarchical structure by decreasing the within-class distance and increasing the between-class distance. Our method aggregates the discriminative deep features in different levels into a hierarchical aggregated deep feature that considers the spatial hierarchy and feature distinctiveness. Lastly, we build a multichannel 1-D convolutional neural network to classify the unknown points. Our tests demonstrate that the proposed hierarchical aggregated deep feature method can enhance point cloud classification results. Comparing with seven state-of-the-art methods, those results also verified the superior performance of our method.

Due to errors in sensors and positioning, there exist mismatches between different phases of mobile laser scanning point clouds, which impede the application of point cloud, such as changing detection and deformation monitoring. To rectify such mismatches, we designed a 3-D deep feature construction method for point cloud registration. The proposed method combines two 3-D convolutional neural networks into a uniform deep learning model to extract 3-D deep features. First, the corresponding points and noncorresponding points are set to train the deep learning model to minimize the distance between corresponding points’ features and maximize the distance between features of noncorresponding points. Second, in the test phase, the 3-D deep feature for each keypoint was extracted by the trained deep learning model. This could be used to determine the corresponding points by the$k$-dimensional tree and random sample consensus (RANSAC) algorithm. Finally, a transformation matrix was calculated based on the corresponding points and was then applied to point cloud registration. The experimental results illustrated that the proposed method of using 3-D deep features is more efficient at a corresponding point search than representatives of three existing methods. It also improved registration accuracy.

This paper introduces a novel methodology for residential building contouring from large-scale airborne point clouds. Unlike other methods that handle linearization and regularization of the linear primitives separately by imposing rigid constraints, we propose an optimization-based linearization and global regularization to form accurate, topologically error-free, and lightweight polygons. To this end, we enhance the classic density-based spatial clustering of applications with noise algorithm to segment individual building entities at the instance level. The initial contours of each individual building are then delineated and further decomposed by a novel topologically aware propagation process and a global optimization technique. The decomposed linear primitives are fed into the global regularization step, from which the regular shapes are learned and enforced hierarchically by imposing constraints, such as parallelism, homogeneity, orthogonality, and collinearity. Based on the concept of hybrid representation, the regularized and unaltered linear primitives are jointly connected in an esthetic way. Various experiments using representative buildings and large-scale residential scenes from the Dutch AHN3 data set have shown that the proposed methodology generates meaningful building contouring representation in terms of accuracy, compactness, topology, and levels of detail abstraction while being robust and scalable.

This paper presents a novel topologically aware 2.5-D building modeling methodology from airborne laser scanning point clouds. The building reconstruction process consists of three main steps: primitive clustering, boundary representation, and geometric modeling. In primitive clustering, we propose an enhanced probability density clustering algorithm to cluster the rooftop primitives by taking into account the topological consistency among primitives. In the second step, we employ a novel Voronoi subgraph-based algorithm to seamlessly trace the primitive boundaries. This algorithm guarantees the production of geometric models without crack defects among adjacent primitives. The primitive boundaries are further divided into multiple linear segments, from which the key points are generated. These key points help to form a hybrid representation of the boundary by combining the projected points with part of the original boundary points. The model representation by the hybrid key points is flexible and well captures the rooftop details to generate lightweight and highly regular building models. Finally, we assemble the primitive boundaries to form the topologically correct entities, which are regarded as the basic units for primitive triangulation. The reconstructed models not only have accurate geometry and correct topology but more importantly have abundant semantics, by which five levels of building models can be generated in real time. The proposed reconstruction method has been comprehensively evaluated on Toronto data set in terms of model compactness, multilevel model representation, and geometric accuracy.

The effective extraction of shape features is an important requirement for the accurate and efficient classification of terrestrial laser scanning (TLS) point clouds. However, the challenge of how to obtain robust and discriminative features from noisy and varying density TLS point clouds remains. This paper introduces a novel multiscale and hierarchical framework, which describes the classification of TLS point clouds of cluttered urban scenes. In this framework, we propose multiscale and hierarchical point clusters (MHPCs). In MHPCs, point clouds are first resampled into different scales. Then, the resampled data set of each scale is aggregated into several hierarchical point clusters, where the point cloud of all scales in each level is termed a point-cluster set. This representation not only accounts for the multiscale properties of point clouds but also well captures their hierarchical structures. Based on the MHPCs, novel features of point clusters are constructed by employing the latent Dirichlet allocation (LDA). An LDA model is trained according to a training set. The LDA model then extracts a set of latent topics, i.e., a feature of topics, for a point cluster. Finally, to apply the introduced features for point-cluster classification, we train an AdaBoost classifier in each point-cluster set and obtain the corresponding classifiers to separate the TLS point clouds with varying point density and data missing into semantic regions. Compared with other methods, our features achieve the best classification results for buildings, trees, people, and cars from TLS point clouds, particularly for small and moving objects, such as people and cars.

A 3-D tree structure plays an important role in many scientific fields, including forestry and agriculture. For example, terrestrial laser scanning (TLS) can efficiently capture high-precision 3-D spatial arrangements and structure of trees as a point cloud. In the past, several methods to reconstruct 3-D trees from the TLS point cloud were proposed. However, in general, they fail to process incomplete TLS data. To address such incomplete TLS data sets, a new method that is based on a structure-aware global optimization approach (SAGO) is proposed. The SAGO first obtains the approximate tree skeleton from a distance minimum spanning tree (DMst) and then defines the stretching directions of the branches on the tree skeleton. Based on these stretching directions, the SAGO recovers missing data in the incomplete TLS point cloud. The DMst is applied again to obtain the refined tree skeleton from the optimized data, and the tree skeleton is smoothed by employing a Laplacian function. To reconstruct 3-D tree models, the radius of each branch section is estimated, and leaves are added to form the crown geometry. The developed methodology has been extensively evaluated by employing a dozen TLS point clouds of various types of trees. Both qualitative and quantitative performance evaluation results have indicated that the SAGO is capable of effectively reconstructing 3-D tree models from grossly incomplete TLS point clouds with significant amounts of missing data.

This article presents a spatial cognition analysis technique for automated urban building clustering based on urban morphology and Gestalt theory. The proximity graph is selected to present the urban mrphology. The proximity graph considers the local adjacency among buildings, providing a large degree of freedom in object displacement and aggregation. Then, three principles of Gestalt theories, proximity, similarity, and common directions, are considered to extract potential Gestalt building clusters. Next, the Gestalt features are further characterized with seven indicators, that is, area difference, height difference, similarity difference, orientation difference, linear arrangement difference, interval difference, and oblique degree of arrangement. A support vector machine (SVM)-based approach is employed to extract the Gestalt building clusters. This approach transforms the Gestalt cluster extraction into a supervised discrimination process. The method presents a generalized approach for clustering buildings of a given street block into groups, while maintaining the spatial pattern and adjacency of buildings during the displacement operation. In applications of urban building generalization and three-dimensional (3D) urban panoramic-like view, the method presented in this article adequately preserves the spatial patterns, distributions, and arrangements of urban buildings. Moreover, the final 3D panoramic-like views ensure the accurate appearance of important features and landscapes.