Uncrewed aerial vehicle (UAV) photogrammetry has been widely applied in urban reconstruction for acquiring 3-D models. However, during the UAV imaging process, there is occlusion on the ground, leading to severe geometric distortion and texture errors in the road areas of the photogrammetric models. Existing methods are unable to automatically obtain more refined repair effects. Considering this, we have proposed a method for automatically repairing photogrammetric models by removing occlusions to enhance mesh quality. The method first used an improved detection approach tailored for small targets in UAV imagery to identify the target location. Based on this, a local shadow adaptive extraction strategy was used to rapidly extract the target’s shadow, mitigating the impact of shadows on the model’s texture. Subsequently, the extracted target and its shadow were fed into the LaMa repair model to complete the texture completion. By detecting the target’s position in the imagery and using back projection, the 3-D vertices of the mesh target were determined. The surrounding mesh points were then used to flatten it, restoring the 3-D mesh to its state before the target was present. The completed texture was then remapped onto the flattened mesh, achieving the automatic repair of the photogrammetric model. The results indicate that the target detection method proposed in this study, on the VisDrone-DET 2019 dataset, shows a clear superiority over other methods in all metrics. Experiments on two real-world UAV datasets demonstrate that, compared to conventional 3-D reconstruction methods, the technical approach proposed in this study can generate more realistic 3-D meshes after automatically removing unwanted targets in photogrammetric model repair.

The depth map estimation from a set of remote sensed images has been a challenging task due to the complexity of the real-world scenes, yet it is of great importance for applications such as 3D reconstruction and digital surface model (DSM). Most of the existing learning based multi-view stereo (MVS) approaches reconstruct depth map through supervised models that are trained with large amounts of data without any prior knowledge about the areas of interest. A recent foundational model Depth Anything, provides a robust method to estimate the relative depth map (RDM) based on single optical image. Leveraging its excellent performance in depth estimation and fine generalization capability, we propose a RDM fusion module that can be integrated with most state-of-the-art (SOTA) learning based MVS frameworks to improve their performance in 3D reconstruction. Extensive experiments are conducted to verify the effectiveness of the proposed module and positive results indicate that the integration of the proposed module leads to better accuracy and completeness compared to the benchmark models. The codes are available at https://github.com/2022hong/RDM-MVS/tree/main.

With the rapid advancements of 3-D acquisition technology, 3-D change detection has gained lots of attentions recently. Existing deep learning-based point cloud change detection methods usually adopt a common encoder-decoder structure to learn pointwise features. However, these feature learning backbones are not specifically designed for change detection task, and ignore the local structure discrepancies during feature learning. To address these issues, this article proposes a multiscale difference-aware network (Ms-DANet) for 3-D point cloud change detection. First, we propose a difference-guided multiscale feature learning (DG-MsFL) module to enhance the feature differences between bi-temporal point clouds at multiple scales during feature encoding, and use these differences to guide the network focusing more on the local structures with large discrepancies. Next, we introduce a multiscale difference feature fusion (Ms-DFF) module to fuse the multiscale feature differences to learn more discriminative features during feature decoding. Finally, we treat the point cloud change detection task as a semantic classification problem, and propose a multiscale loss (Ms-Loss) function to promote the network training. We conduct experiments on the real-world street-level point cloud change detection dataset SLPCCD and the simulated airborne urban point cloud change detection dataset URB3DCD. The experimental results show that Ms-DANet obtains a significant improvement on both the real-world and simulated point cloud change detection datasets, demonstrating its effectiveness and robustness across various sensors and data modalities.

Point cloud oversegmentation method can obtain a series of superpoints by grouping points that are semantically and geometrically consistent. The generated superpoints can be treated as the basic processing units in various downstream tasks to improve task performance and processing efficiency. However, due to the high semantic and geometric complexity of point cloud scenes, obtaining high-quality superpoints is still challenging. Aiming to generate high-quality indoor superpoints, we propose an end-to-end supervised contrastive learning framework SCL-OverSeg for indoor point cloud oversegmentation. Firstly, to solve the challenge of balancing the importance of geometric similarity and spatial proximity constraint between points and superpoints in indoor scenes, we integrate the geometric similarity and spatial proximity constraint into the supervision signal by generating the superpoint ground truth. To solve the challenge of superpoints crossing objects, we propose to utilize instance labels rather than semantic labels to generate the ideal superpoint ground truth as the object-level supervision signal. Secondly, to construct the distinguishable embedding space facilitating to the assignments of points to superpoints, we propose point-superpoint contrastive learning to compel the network to project each point to be closer to the reasonable superpoint in embedding space. Besides, with the instance labels, to improve the superpoint performance on object boundaries, we propose the object boundary contrastive learning to enhance the feature distinguishability between tough points across the object boundaries. Extensive experiments demonstrate that SCL-OverSeg can effectively improve indoor oversegmentation performance, especially on object boundaries. The relevant codes will be available onhttps://github.com/sssssyf/SCL-OverSeg.

Abstract Point cloud registration is of great significance to the reconstruction of high‐precision 3D city models. There are some challenges when aligning aerial and close‐range photogrammetric point clouds, such as huge view differences caused by the different sights of the sensors, massive noisy points due to the error in dense matching, and scale uncertainty since there are no control points for absolute orientation. To achieve complementary advantages of aerial and close‐range point clouds, in this paper, a robust cross‐source point clouds registration method is proposed using image visual context features and scale consistency. First, a cross‐view image matching method based on image visual context features is proposed to obtain corresponding points. Second, to overcome the challenges of noisy points and scale differences, an outlier filtering method is designed based on scale consistency. Finally, the dual quaternions model considering the scale factor is introduced to solve the spatial transformation model. To analyze the feasibility of this method qualitatively and quantitatively, experiments are conducted using the public scene dataset of Dortmund, Germany, and the scene dataset of Zhengzhou City, China. Three kinds of cross‐source point clouds registration experiments are conducted in this paper, including aerial and close‐range point clouds registration in Dortmund, and aerial and ground point clouds registrations in both Dortmund and Zhengzhou. The chamfer distances of the three sets of experiments are 4.48 m, 5.97 m and 4.78 m, respectively. The ablation study shows that the outlier filtering method and the dual quaternions model improve the accuracy by at least 14% and 25%, respectively. The experiments demonstrate that the method accomplishes the cross‐source point clouds registration in large‐scale scenes accurately and efficiently, providing a solid foundation for subsequent fine 3D reconstruction.

In this letter, we introduce MappingConvSeg, a continuous convolution network for semantic segmentation of large-scale point clouds. In particular, a conceptually simple, end-to-end learnable, and continuous convolution operator is proposed for learning spatial correlation of unstructured 3-D point clouds. For each local point set, the unstructured point features are first mapped onto a series of learned kernel points based on the spatial relationship, and the continuous convolution is then applied to capture specific local geometrical patterns. Taking the proposed mapping convolution operation as the building block, a hierarchical network is then built for large-scale point cloud semantic segmentation. Experimental results conducted on two public benchmarks, including Toronto-3D and Stanford large-scale 3-D Indoor Spaces (S3DIS) dataset, demonstrate the superiority of the proposed method.

Generally, purified algorithms of image matching points use some of the points as initial input. For the algorithms, as the purification results are quite easy to fall into a local optimum, they usually have such problems as rejecting some of correct matching points. To solve this problem, we introduce singular value decomposition model, which take the whole matches as input to obtain a more accurate result through iterative robust solving. Extensive experiments on practical images demonstrate the excellent performance of our proposed method.