ABSTRACT Point cloud registration, a core task in 3D computer vision for aligning two point clouds via rotation and translation, underpins critical applications like robotic navigation and 3D reconstruction. Classical methods (e.g., Iterative Closest Point) easily converge to local minima under poor initial alignment. Deep learning–based approaches, while efficient, suffer from high annotation costs for large‐scale data. Existing reinforcement learning (RL)‐based methods rely on simple PointNet feature extractors, which are insensitive to local geometric details and thus yield suboptimal registration precision. To address these challenges, we propose ACT‐Agent: Affinity‐Cross Transformer for point cloud registration via reinforcement learning, a novel method that formulates point cloud registration as an RL Markov decision process for iterative optimisation. We leverage Pointnet and Affinity‐Cross Transformer to extract and enhance expressive salient features and assign adaptive weights to channels based on their relative importance. We use RL to autonomously learn from feedback in the environment, freeing ourselves from dependence on data annotation. Experimental results on ModelNet40 (synthetic data) and ScanObjectNN (real‐world data) demonstrate that our proposed ACT‐Agent achieves higher accuracy, efficiency, and generalisation ability than the state‐of‐the‐art methods of point cloud registration.

Object detection is a fundamental challenge encountered in understanding and analyzing remote sensing images. Many current detection methods struggle to identify objects in remote sensing images due to poor feature saliency, diverse directions, and unique viewing angles of small remote sensing targets. As most remote sensing images are captured through overhead photography, the structural contour features of objects in these images remain relatively stable, such as the cross-shaped structure of the aircraft and the rectangular structure of the vehicle. Therefore, we utilize the physical simulation images as prior knowledge to supplement the invariant structural features of the target, extract the target’s key geometric features through the dynamic matching method, and fuse it with the features extracted by the neural network to detect the remote sensing small target more effectively in this article. We propose a template matching method based on physical simulation images (TMSI) and add it to the modified Darknet-53 (named TMSI-Net) for small target detection. Based on this, we perform specific transformations on existing templates in terms of target scale and direction to achieve better adaptation to the corresponding goals and propose DTMSI-Net with a dynamic template library (Dynamic TMSI-Net). Experiments on several datasets demonstrate that the DTMSI-Net exhibits higher detection accuracy and more stable performance compared with the state-of-the-art methods, 4% and 2.7% higher than the second-ranked model on the VEDAI and NWPU datasets, respectively.

Learning-based multi-view stereo (MVS) is gaining prominence as a method for 3D reconstruction. However, existing methods in the process of feature learning fail to focus on the structural information implied in the scene. This oversight prevents the network from perceiving the geometric properties of the scene and weakens the generalizability of the network. Therefore, we propose a novel framework named Point Feature Relation Network for Multi-view Stereo (PFR-MVSNet), which is composed of a Dynamic Structure Perception (DSP) module, an Adaptive Feature Exploration (AFE) module, and a Point Transformer Block (PTB) module, to solve the problems caused by the oversight. The DSP module first augments the feature of the 3D point cloud from multi-view 2D features, then establishes spatial structure relations within local regions on the point cloud and guides the feature learning of points through the aggregated structure information. After the network has fully learned the intra-region structure features, the AFE module repartitions perception regions with similar features. The point features within the regions are further learned by the PTB module. We evaluate our method on three benchmark datasets: DTU, Tanks & Temples, and ETH3D. The experimental results show that our method achieves superior accuracy of 0.289 mm on the DTU dataset and exhibits more robust generalization on the Tanks & Temples and ETH3D datasets compared with other learning-based MVS methods.

In order to retain more feature information of local areas on a point cloud, local grouping and subsampling are the necessary data structuring steps in most hierarchical deep learning models. Due to the disorder nature of the points in a point cloud, the significant time cost may be consumed when grouping and subsampling the points, which consequently results in poor scalability. This paper proposes a fast data structuring method called PSNet (Point Structuring Net). PSNet transforms the spatial features of the points and matches them to the features of local areas in a point cloud. PSNet achieves grouping and sampling at the same time while the existing methods process sampling and grouping in two separate steps (such as using FPS plus kNN). PSNet performs feature transformation pointwise while the existing methods uses the spatial relationship among the points as the reference for grouping. Thanks to these features, PSNet has two important advantages: 1) the grouping and sampling results obtained by PSNet is stable and permutation invariant; and 2) PSNet can be easily parallelized. PSNet can replace the data structuring methods in the mainstream point cloud deep learning models in a plug-and-play manner. We have conducted extensive experiments. The results show that PSNet can improve the training and inference speed significantly while maintaining the model accuracy.

A model segment method called Octree Subdivision has been presented for long years, which allows any three-dimensional point cloud object to be subdivided into infinitesimals so that it can be approximated by a particular surface function. In this paper, we proposed a new method named self-merging octree to reconstruct the surface of 3D Point Cloud which can be obtained by laser scanners or generated by some 3D modeling software. Different from any other surface reconstruction algorithms such as local property-based or specific type-based, a function pool-based was introduced in our research because it can express many different types of surfaces. We subdivide point cloud model by self-merging octree and categorize it by the neuro-network. In this idea, it is easy for us to find a proper surface function to present the subsurface of the model. What‘s more, while we extend the function pool, we can indicate far more style models. We have tried to reconstruct many point cloud models‘ surfaces in this way, and it works well and also shows its potential ability to build a bridge in the fields of model editing, model splicing, and model deformation.