We propose a Digital Building Analysis (DBA), a digital system for building-scale cloud-based data integration and data analytics. By connecting to cloud mapping platforms such as Google Map Platforms APIs, by leveraging state-of-the-art multi-agent Large Language Models data analysis using ChatGPT(4o) and Deepseek-V3/R1, and by using our Gaussian Splatting-based mesh extraction pipeline, our framework can retrieve a building’s 3D model, visual descriptions, and achieve cloud-based mapping integration with large language model-based data analytics using a building’s address, postal code, or geographic coordinates, and be easily extended to perform data analysis on other cloud-based data streams.

Three-dimensional urban scene reconstruction and modeling is a crucial research area. From a technical perspective, it is an interdisciplinary research area spanning computer vision, computer graphics, and photogrammetry. Its applications span across multiple disciplines including autonomous navigation with 3-D scene understanding, remote sensing/photogrammetry for the creation of 3-D maps from aerial/drone/satellite images, geographic information systems with urban digital twins, augmented and virtual reality with photorealistic scene reconstructions. Using Google Earth imagery, we create a 3-D Gaussian splatting (3DGS) model of the Waterloo region centered on the University of Waterloo, and are able to achieve view-synthesis results far exceeding previous 3-D view-synthesis results based on neural radiance fields (NeRFs)which we demonstrate in our benchmark. We also retrieve the 3-D geometry of the scene using the 3-D point cloud extracted from the 3DGS model, thereby reconstructing both the 3-D geometry and photorealistic lighting of the large-scale urban scene.

LiDAR point cloud segmentation has increasingly benefited from the application of Mamba-based models. However, unordered and irregular natures of point clouds necessitates serialization, which significantly impacts the performance of Mamba-based methods. This paper explores the critical role of token serialization in Mamba-based point cloud processing, using the pure Mamba network, PointMamba, as the baseline. We systematically investigated existing point cloud serialization methods, evaluating their performance on two challenging LiDAR datasets: the airborne MultiSpectral LiDAR (MS-LiDAR) dataset and the aerial DALES dataset. To explore the inherent factors of serialization contributing to Mamba’s performance, we design novel indicators for serialization quality, focusing on spatial and semantic proximity. These indicators are validated across all datasets, offering a valuable reference and guidance for advancing token serialization in Mamba-based point cloud processing. Guided by these indicators, we proposed a new point cloud serialization method that integrates spatial and semantic features through a weighted comprehensive distance matrix. The proposed method achieves superior accuracy on both LiDAR datasets, surpassing existing approaches, and establishes a strong foundation for advancing Mamba-based point cloud processing.

Although the application of Transformers to 3-D point cloud processing has achieved significant progress and success, it is still challenging for existing 3-D Transformer methods to efficiently and accurately learn both valuable global and local features for improved applications. This article presents a novel point cloud representational learning network, called 3-D Dual Self-attention global local (GLocal) Transformer Network (3DGTN), for improved feature learning in both classification and segmentation tasks, with the following key contributions. First, a GLocal feature learning (GFL) block with the dual self-attention mechanism [i.e., a novel point-patch self-attention, called PPSA, and a channel-wise self-attention (CSA)] is designed to efficiently learn the global and local context information. Second, the GFL block is integrated with a multiscale Graph Convolution-based local feature aggregation (LFA) block, leading to a GLocal information extraction module that can efficiently capture critical information. Third, a series of GLocal modules are used to construct a new hierarchical encoder–decoder structure to enable the learning of information in different scales in a hierarchical manner. The proposed framework is evaluated on both classification and segmentation datasets, demonstrating that the proposed method is capable of outperforming many state-of-the-art methods on both synthetic and LiDAR data. Our code has been released athttps://github.com/d62lu/3DGTN.

The acquisition of both local and global features from irregular point clouds is crucial for 3D object detection (3DOD). Current mainstream 3D detectors neglect significant local features during pooling operations or disregard many global features of the overall scene context. This paper proposes new techniques for simultaneously learning local-global features of scene point clouds to enhance 3DOD. Specifically, we propose an efficient 3DOD network in indoor point clouds, named SimLOG, which utilizes simultaneous local-global feature learning. SimLOG has two main contributions: a Dynamic Points Interaction (DPI) module to recover local features lost during pooling, and a Global Context Aggregation(GCA) module to aggregate multi-scale features from various layers of the encoder to improve scene context awareness. Unlike traditional local-global feature learning methods, our DPI and GCA modules are integrated into a single feature learning module, making it easily detachable and able to be incorporated into existing 3DOD networks to enhance their performance. SimLOG demonstrates superior performance over twenty competitors in terms of detection accuracy and robustness on both the SUN RGB-D and ScanNet V2 datasets. Specifically, SimLOG boosts the baseline VoteNet by 8.1% of [email protected] on ScanNet V2 and by 3.9% of [email protected] on SUN RGB-D. Code is publicly available athttps://github.com/chenbaian-cs/SimLOG.

This paper applied a transformer based deep learning model 3D Point Cloud Transformer (3DPCT) to conduct a tree species classification of Airborne LiDAR data. There are a total 1291 single tree point clouds of 11 different species from coniferous and deciduous used in this paper. The model integrated the local and global feature learning modules from both pointwise and channel-wise, which provide promising results of tree species classification. We also investigate by adding more channels the classification results can be improved. Different number of points per each sample as the model input also deliver different accuracy. The highest overall accuracy of 11 categories classification achieved 86.1%, and precision and recall of each category provide more directions of future study.

Abstract The intricacy of 3D surfaces often results cutting‐edge point cloud denoising (PCD) models in surface degradation including remnant noise, wrongly‐removed geometric details. Although using multi‐scale patches to encode the geometry of a point has become the common wisdom in PCD, we find that simple aggregation of extracted multi‐scale features can not adaptively utilize the appropriate scale information according to the geometric information around noisy points. It leads to surface degradation, especially for points close to edges and points on complex curved surfaces. We raise an intriguing question – if employing multi‐scale geometric perception information to guide the network to utilize multi‐scale information, can eliminate the severe surface degradation problem? To answer it, we propose a Multi‐offset Denoising Network (MODNet) customized for multi‐scale patches. First, we extract the low‐level feature of three scales patches by patch feature encoders. Second, a multi‐scale perception module is designed to embed multi‐scale geometric information for each scale feature and regress multi‐scale weights to guide a multi‐offset denoising displacement. Third, a multi‐offset decoder regresses three scale offsets, which are guided by the multi‐scale weights to predict the final displacement by weighting them adaptively. Experiments demonstrate that our method achieves new state‐of‐the‐art performance on both synthetic and real‐scanned datasets. Our code is publicly available at https://github.com/hay-001/MODNet .

Point cloud classification is a fundamental task in 3D applications. However, it is challenging to achieve effective feature learning due to the irregularity and unordered nature of point clouds. Lately, 3D Transformers have been adopted to improve point cloud processing. Nevertheless, massive Transformer layers tend to incur huge computational and memory costs. This paper presented a novel hierarchical framework that incorporated convolutions with Transformers for point cloud classification, named 3D Convolution-Transformer Network (3DCTN). It combined the strong local feature learning ability of convolutions with the remarkable global context modeling capability of Transformers. Our method had two main modules operating on the downsampling point sets. Each module consisted of a multi-scale local feature aggregating (LFA) block and a global feature learning (GFL) block, which were implemented by using the Graph Convolution and Transformer respectively. We also conducted a detailed investigation on a series of self-attention variants to explore better performance for our network. Various experiments on ModelNet40 and ScanObjectNN datasets demonstrated that our method achieves state-of-the-art classification performance with a lightweight design. The code is publicly available athttps://github.com/d62lu/3DCTN.

Hough voting, as has been demonstrated in VoteNet, is effective for 3D object detection, where voting is a key step. In this paper, we propose a novel VoteNet-based 3D detector with vote enhancement to improve the detection accuracy in cluttered indoor scenes. It addresses the limitations of current voting schemes, i.e., votes from neighboring objects and background have significant negative impacts. Before voting, we replace the classic MLP with the proposed Attentive MLP (AMLP) in the backbone network to get better feature description of seed points. During voting, we design a new vote attraction loss (VALoss) to enforce vote centers to locate closely and compactly to the corresponding object centers. After voting, we then devise a vote weighting module to integrate the foreground/background prediction into the vote aggregation process to enhance the capability of the original VoteNet to handle noise from background voting. The three proposed strategies all contribute to more effective voting and improved performance, resulting in a novel 3D object detector, termed VENet. Experiments show that our method outperforms state-of-the-art methods on benchmark datasets. Ablation studies demonstrate the effectiveness of the proposed components.