Keypoint detection and descriptor matching are two vital steps in the 3D feature extraction framework, but they are difficult to learn in an end-to-end fashion due to their inherent discreteness. To tackle the non-differentiable operations, we formulate feature extraction as a decision-making problem: the network is treated as a policy pool that can make probabilistic estimations for keypoint selection and feature matching, supervised by maximizing a reward expectation of actions. In this way, we propose a novel end-to-end training paradigm of 3D feature extraction based on the stochastic policy gradient method, named Reinforced Detectors and Descriptors (RDD). Firstly, we propose a local-to-global probabilistic keypoint selection module that formulates the sampling probabilities of keypoints in a local-and-global mechanism to yield sparse and accurate keypoints. Secondly, we regard feature matching as an optimal transport problem and an efficient Sinkhorn method is leveraged to solve the optimal matching probabilities. In particular, we carefully design a reward function and derive gradients of probabilistic actions, thus overcoming the discreteness and providing reinforced supervision signals. Since our reward function is calculated from sampled keypoints rather than from randomly sampled points as in existing methods, the gap between training and inference is bridged. Experimental results demonstrate that our approach exceeds the quality of state-of-the-art methods and shows strong generalization ability. Remarkably, our approach can achieve significantly higher Registration Recall than other advanced methods when aligning scenes with a small number of keypoints, due to our highly accurate and repeatable detector.

Fragment assembly involves restoring broken objects to their original geometries, and has many applications, such as archaeological restoration. Existing learning based frameworks have shown potential for solving part assembly problems with semantic decomposition, but cannot handle such geometrical decomposition problems. In this work, we propose a novel assembly framework, proxy level hybrid Transformer, with the core idea of using a hybrid graph to model and reason complex structural relationships between patches of fragments, dubbed as proxies. To this end, we propose a hybrid attention module, composed of intra and inter attention layers, enabling capturing of crucial contextual information within fragments and relative structural knowledge across fragments. Furthermore, we propose an adjacency aware hierarchical pose estimator, exploiting a decompose and integrate strategy. It progressively predicts adjacent probability and relative poses between fragments, and then implicitly infers their absolute poses by dynamic information integration. Extensive experimental results demonstrate that our method effectively reduces assembly errors while maintaining fast inference speed. The code is available at https://github.com/521piglet/PHFormer.

After decades of investigation, point cloud registration is still a challenging task in practice, especially when the correspondences are contaminated by a large number of outliers. It may result in a rapidly decreasing probability of generating a hypothesis close to the true transformation, leading to the failure of point cloud registration. To tackle this problem, we propose a transformation estimation method, named Hunter, for robust point cloud registration with severe outliers. The core of Hunter is to design a global-to-local exploration scheme to robustly find the correct correspondences. The global exploration aims to exploit guided sampling to generate promising initial alignments. To this end, a hypergraph-based consistency reasoning module is introduced to learn the high-order consistency among correct correspondences, which is able to yield a more distinct inlier cluster that facilitates the generation of all-inlier hypotheses. Moreover, we propose a preference-based local exploration module that exploits the preference information of top- k promising hypotheses to find a better transformation. This module can efficiently obtain multiple reliable transformation hypotheses by using a multi-initialization searching strategy. Finally, we present a distance-angle based hypothesis selection criterion to choose the most reliable transformation, which can avoid selecting symmetrically aligned false transformations. Experimental results on simulated, indoor, and outdoor datasets, demonstrate that Hunter can achieve significant superiority over the state-of-the-art methods, including both learning-based and traditional methods (as shown in Fig. 1). Moreover, experimental results also indicate that Hunter can achieve more stable performance compared with all other methods with severe outliers.

We propose a robust algorithm for aligning rigid, noisy, and partially overlapping red green blue-depth (RGB-D) point clouds. To address the problems of data degradation and uneven distribution, we offer three strategies to increase the robustness of the iterative closest point (ICP) algorithm. First, we introduce a salient object detection (SOD) method to extract a set of points with significant structural variation in the foreground, which can avoid the unbalanced proportion of foreground and background point sets leading to the local registration. Second, registration algorithms that rely only on structural information for alignment cannot establish the correct correspondences when faced with the point set with no significant change in structure. Therefore, a bidirectional color distance (BCD) is designed to build precise correspondence with bidirectional search and color guidance. Third, the maximum correntropy criterion (MCC) and trimmed strategy are introduced into our algorithm to handle with noise and outliers. We experimentally validate that our algorithm is more robust than previous algorithms on simulated and real-world scene data in most scenarios and achieve a satisfying 3-D reconstruction of indoor scenes.

In this paper, we propose an accurate global registration (TAG-Reg) algorithm for poor initialization and partially overlapping point clouds registration problem. Firstly, methods based on geometric structure information of points can get the accurate results, which is vulnerable to poor initialization. Meanwhile, existing features based global methods can solve poor initialization problem at a certain extent, but it cannot obtain accurate results. So, we combine the geometric structure information with feature as hybrid feature to solve poor initialization problem completely and obtain accurate results. Secondly, we introduce dynamic trimmed strategy combining with hybrid feature to deal with partially overlapping problem. Then, to improve the accuracy of our method, we utilize the probabilistic method to suppress noise. At last, we establish the TAG-Reg model and propose an iterative algorithm to solve this problem. Experimental results show that our TAG-Reg achieves state-of-the-art performance compared to existing non-deep learning and recent deep learning methods. Our source code will open at https://github.com/BiaoBiaoLi/TAG-Reg.

In this paper, we propose a deep weight global registration (DWG-Reg) algorithm for poor initialization and partially overlapping point clouds registration problem. Our DWG-Reg is based on three modules: a bidirectional nearest search strategy for correspondence, a convolutional network for correspondence confidence prediction which consists of Hybird Distance Generator, optimal annealing Parameter Prediction network and a robust kernel function, a weighted optimizer algorithm for closed-form pose estimation. Experimental results show that our DWG-Reg achieves state-of-the-art performance compared to existing non-deep learning and recent deep learning methods. Our source code will open at https://github.com/BiaoBiaoLi/DWG-Reg.

Abstract This paper proposed a novel precise point set registration method based on feature fusion for three-dimensional data. Firstly, for the prominent foreground with dense and continuous cluster structure, we propose an automatic extraction method combining the principal component analysis projection and density-based clustering method. Secondly, for point sets containing noises, we introduce correntropy measurement into registration to weaken their influence. Thirdly, for the precise registration of uneven distribution of points in the same point set, we propose a feature fusion based algorithm which is distribution specific, using point-to-point measurement for densely distributed foreground and point-to-plane measurement for sparsely distributed background, in case that only one measurement method is used for the whole point set the registration gets trapped into local extremum. Finally, we give the optimization algorithm of the proposed method. We conduct experiments on real orthodontics scenes to verify the effectiveness of our proposed feature extraction method and registration algorithm, and experimental results demonstrate that both the proposed solutions are proper for their respective tasks than other existing methods.

Palms alignment is an important work for palmprint recognition in uncontrolled environment. Many methods have made progress to achieve alignment. But most of them ignore the palm's angles, which could not satisfy the alignment initialization when the hand has a large in-plane rotation. In this paper, we propose a palms alignment with affine transformation method based on a two-stage convolutional neural network (CNN). The basic idea is to rotate the target palm into the same angle category to avoid the following affine registration has a big matching error at the beginning. At the stage I, the given target palm is classified into two angle categories. At the stage II the upside down palm is firstly rotated 180 degrees, and then inputted into the subsequent feature extraction network, feature matching layer and regression network to achieve the affine alignment. Experimental results have proved the effectiveness of our method.

In this study, the authors present a novel individual retrieval method based on oral cavity point cloud data and correntropy‐based registration algorithm. Since the three‐dimensional oral cavity data contains a large amount of noise and outliers, it may lead to a decrease in registration accuracy, which affects the accuracy of retrieval rate. Therefore, the authors introduce the correntropy into the rigid registration algorithm to solve this problem. Then, they filter the matched point cloud data and then use the mean squared error to judge the individual differences of the model data. Finally, the accurate retrieval of the oral cavity data is realised. Experimental results demonstrate the proposed retrieval three‐dimensional model algorithm can be successfully searched under different model data, which can help forensics use the characteristics of biological individuals to accurately search and identify, and improve recognition efficiency.

The coronavirus disease 2019 (COVID-19) breaking out in late December 2019 is gradually being controlled in China, but it is still spreading rapidly in many other countries and regions worldwide. It is urgent to conduct prediction research on the development and spread of the epidemic. In this article, a hybrid artificial-intelligence (AI) model is proposed for COVID-19 prediction. First, as traditional epidemic models treat all individuals with coronavirus as having the same infection rate, an improved susceptible-infected (ISI) model is proposed to estimate the variety of the infection rates for analyzing the transmission laws and development trend. Second, considering the effects of prevention and control measures and the increase of the public's prevention awareness, the natural language processing (NLP) module and the long short-term memory (LSTM) network are embedded into the ISI model to build the hybrid AI model for COVID-19 prediction. The experimental results on the epidemic data of several typical provinces and cities in China show that individuals with coronavirus have a higher infection rate within the third to eighth days after they were infected, which is more in line with the actual transmission laws of the epidemic. Moreover, compared with the traditional epidemic models, the proposed hybrid AI model can significantly reduce the errors of the prediction results and obtain the mean absolute percentage errors (MAPEs) with 0.52%, 0.38%, 0.05%, and 0.86% for the next six days in Wuhan, Beijing, Shanghai, and countrywide, respectively.

Basal cell carcinoma is very similar to the clinical traits of seborrheic keratosis, which is still a difficult problem in medical image analysis. To accurately classify it, this paper proposes a multi-model ensemble method based on the maximum correntropy criterion (MCC) and convolutional neural network (CNN). First of all, it is well known that the CNN single models like ResNet, Xception, DensNet, etc. have a good effect on the classification, but the accuracy is still limited, so the multi-model ensemble method is presented to improve the accuracy. Secondly, the traditional multi-model ensemble methods, such as voting and linear regression, can improve the accuracy of the model, but it means that the weight computation of each model does not consider the noise, and could not obtain good results. Therefore, we propose the MCC for the model ensemble, which overcomes the noise in the data and effectively improves the classification accuracy. Finally, our proposed multi-model ensemble algorithm based on the MCC achieved an accuracy of 97.07% in the basal cell carcinoma and seborrheic keratosis classification experiments, surpassing the CNN single model and traditional multi-model ensemble method.

The traditional iterative closest point (ICP) algorithm could register two points sets well, but it is easily affected by local dissimilar. To deal with this problem, this paper proposes an isotropic scaling ICP algorithm with corner point constraint. First, an objective function is proposed under the guidance of the corner points, as the corner points can preserve the similar of the whole shapes. Secondly, a new ICP algorithm is used to complete the isotropic scaling registration. At each step of this new algorithm, the correspondence is built based on the closest point searching, and then a closed-form solution of the transformation is computed. The experimental results demonstrate that our algorithm can prevent the influence of the local dissimilar and improve the registration precision compared with the traditional ICP algorithm.

The traditional affine iterative closest point (ICP) algorithm is fast and accurate for affine registration between two point sets, but it is easy to fall into local minimum. This paper proposes a robust Affine ICP algorithm based on corner points. First, an objective function is established under the guidance of corner points, where the corner points as the shape control point guides the affine registration of 2D point sets. Then, at each step of the algorithm, the affine transformation obtained by the last iterative step is used to establish the correspondence of these two point sets. Next, the new affine transformation is solved by using the objective function under the guidance of the corner points. Experimental results demonstrate that the robustness and convergence of our algorithm are greatly improved compared with the traditional affine registration algorithm.