Non-rigid registration is essential for augmented reality-guided laparoscopic liver surgery, as it enables the fusion of preoperative information such as tumor location and vascular structures into the limited intraoperative view, thereby enhancing surgical navigation. A prerequisite is the accurate prediction of intraoperative liver deformation, which remains highly challenging due to factors such as large deformation caused by pneumoperitoneum, respiration and tool interaction as well as noisy intraoperative data, and limited field of view due to occlusion and constrained camera movement. To address these challenges, we introduce PIVOTS, a Preoperative to Intraoperative VOlume-To-Surface registration neural network that directly takes point clouds as input for deformation prediction. The geometric feature extraction encoder allows multi-resolution feature extraction, and the decoder, comprising inter-modality cross attention modules, enables information exchange between pre- and intraoperative features and accurate multi-level displacement prediction. We train the neural network on a large synthetic dataset created using a biomechanical simulation pipeline that explicitly targets the mentioned intraoperative challenges and validate its performance on both synthetic and real datasets. Results demonstrate superior registration performance of our method compared to baseline methods, exhibiting strong robustness against high amounts of noise, large deformation, and various levels of intraoperative visibility. The network is fast enough to run multiple times per second and directly generalizes to new patients without retraining. We publish training and test sets as evaluation benchmarks in an effort to contribute to the development of more robust liver registration methods based on volume-to-surface data. Code, docker container and datasets are available athttps://github.com/pengliu-nct/PIVOTS.

In computer-assisted orthopedic surgery (CAOS), robust and accurate registration of the preoperative full bone model and the intraoperative partial point set is a prerequisite for reliable surgical navigation, yet remains highly challenging due to partial overlap, noise, and outliers. We propose the Overlap-Guided Bidirectional Hybrid Mixture Registration (OBHMR) framework for robust and accurate partial-to-full registration. First, geometric features (i.e., surface normals) extracted from raw point sets are incorporated in both correspondence estimation and transform computation. Meanwhile, we formulate a hybrid mixture model that jointly represents positions with Gaussian mixtures (GMMs) and normals with von Mises–Fisher (vMF) mixtures across the two generalized point sets. Second, a dual-branch overlap prediction network leverages feature similarity and geometric structure to provide accurate point-wise overlap scores that guide hybrid-mixture construction under partial overlap. Third, a correspondence module integrates rotation-invariant features, multi-level self-attention, and clustering-based refinement to enhance reliability under noise and misalignment. Finally, a bidirectional objective jointly aligns source-to-target and target-to-source mixtures, explicitly accounting for discrepancies induced by noise and outliers in both the preoperative and intraoperative point sets to achieve robust optimization. Extensive experiments on 1,399 femur and 1,301 hip models demonstrate superior performance over state-of-the-art methods across overlap ratios from 5% to 70%, under both isotropic and anisotropic noise and outlier ratios up to 100%, achieving errors as low as 1.27° rotation and 1.18 mm translation at 50% overlap with 2.5mm noise. Additional tests on liver and ModelNet40 confirm strong generalization across medical and non-medical data. Ablation studies further validate the contributions of normals, overlap estimation, and the bidirectional formulation.

This paper presents a novel robust and accurate normal-assisted learning-based rigid point set registration approach, i.e., Deep Bi-directional Hybrid Mixture Registration (DeepBHMR), where normal vectors are used in both correspondence and transformation computational stages while the bi-directional registration processes are considered. DeepBHMR consists of three components, (1) the correspondence estimation network that predicts the correspondence probabilities; (2) the posterior estimation module that computes the HMMs parameters; (3) the transformation estimation module that calculates the rigid transformation matrix by utilizing the bidirectional optimization mechanism. DeepBHMR has been extensively validated on various medical data sets, outperforming state-of-the-art registration methods. For femur bones, the mean rotation error value is approximately 1° (i.e, 1.01°) and the translation error is less than 1 mm (i.e., 0.30 mm) respectively, which meets the requirement of computer-assisted orthopedic surgery. Furthermore, even (1) trained with femur data and tested on distinct shapes and (2) under the large transformation, the mean RMSE values of registration are 2.60 mm and 3.05 mm respectively, demonstrating DeepBHMR’s favorable generalizability to different data shapes and great capability to handle global registration. Additionally, the individual significant contributions and computational efficiency of adopting normal vectors and utilizing the bidirectional mechanism have been validated in ablation studies. The results demonstrate the DeepBHMR’s favorable generalizability from femur bones to hip bones and that DeepBHMR can successfully handle the large transformation or partial-to-full registration simultaneously. The code implementation of DeepBHMR has been made publicly available at https://github.com/zzyrobot/DeepBiHMM.git.

Biomechanical modelling of soft tissue provides a method for constraining medical image registration, such that the estimated spatial transformation is considered biophysically plausible. Existing methods either directly optimize the loss function containing the biomechanical-constrained regularization term over deformations, which takes much computational time, or are trained using biomechanically plausible data generated via finite element simulation, which is cumbersome. This work first instantiates the recently-proposed physics-informed neural networks (PINNs) to 3D elastic models that are used to establish the partial differential equations (PDEs) representing physics laws of biomechanical constraints to be satisfied. The registration algorithm that aligns point sets considering PINN-imposed biomechanics (i.e., the forward problem) is then formulated. In addition, the inverse problem and its algorithm of physical parameter (i.e., material property) estimation along with the registration are also formulated and developed. We carefully compare linear and nonlinear elasticity theories' capabilities in solving both tasks of forward registration and inverse physical parameter identification under PINNs respectively. Furthermore, two specific network configurations that leverage one common branch or two individual branches to predict deformation vectors and biomechanical states are also constructed and compared. The proposed PINNs-based registration approaches have been extensively evaluated with three experiments, that is single and multiple patient MRI-US registration using clinical MRI-US pairs, and registration using pairs of undeformed MR images from clinical cases of prostate cancer biopsy and deformed counterparts with finite-element-computed ground-truth deformation. Results demonstrate that the proposed methods achieve state-of-the-art performances compared to biomechanical-model-based and learning-based registration approaches, and the biomechanical constraints of soft tissues have been successfully warranted after registration. The codes are available at https://github.com/ZheMin-1992/Registration_PINNs.

In recent times, geometric foundation models have demonstrated remarkable performance in depth estimation tasks, benefiting from exposure to large-scale data that enables the learning of intricate geometric structures and spatial dependencies. However, their large parameter sizes and high computational complexity pose significant challenges in meeting the efficiency requirements of downstream surgical applications. Consequently, the design of a high-performance yet lightweight monocular depth estimator has become a focal point of research. To this end, we harness the rich geometric priors encoded in geometric foundation models and introduce a novel trinity distillation scheme that transfers geometric knowledge across three complementary dimensions, namely spatial, spectral and gradient, into a compact depth estimator. To further enhance prediction quality, we develop a semantic distribution alignment strategy to effectively suppress pseudo-texture artifacts arising from the limited semantic representation capability of the lightweight estimator. Extensive experiments on the SCARED, SERV-CT, Hamlyn, and C3VD datasets demonstrate that the proposed method either surpasses or achieves comparable performance to previous state-of-the-art competitors, with a smaller model size and reduced computational overhead. Code will be available at: https://github.com/ShuweiShao/LiteNet.

Robust and accurate 2D/3D registration, which aligns preoperative models with intraoperative images of the same anatomy, is crucial for successful interventional navigation. To mitigate the challenge of a limited field of view in single-image intraoperative scenarios, multi-view 2D/3D registration is required by leveraging multiple intraoperative images. In this paper, we propose a novel multi-view 2D/3D rigid registration approach comprising two stages. In the first stage, a combined loss function is designed, incorporating both the differences between predicted and ground-truth poses and the dissimilarities (e.g., normalized cross-correlation) between simulated and observed intraoperative images. More importantly, additional cross-view training loss terms are introduced for both pose and image losses to explicitly enforce cross-view constraints. In the second stage, test-time optimization is performed to refine the estimated poses from the coarse stage. Our method exploits the mutual constraints of multi-view projection poses to enhance the robustness of the registration process. The proposed framework achieves a mean target registration error (mTRE) of$0.79+2.17\ \mathbf{mm}$on six specimens from the DeepFluoro dataset, demonstrating superior performance compared to state-of-the-art registration algorithms.

In computer-assisted orthopedic surgery (CAOS), accurate point set registration is essential for enhancing surgical accuracy. However, the sparse and low-overlap nature of intraoperative point sets presents significant challenges for reliable registration. To deal with these challenges, we propose a novel registration-after-completion framework, where the intraoperative point set is first completed, after which the two full point sets are registered. Our main contributions include the follows. First, we propose a progressive two-stage strategy to progressively complete the sparse and partial intraoperative point set. Second, considering that 1) intra-operative point set contains noise 2) the point completion process is not perfect, and 3) the resolution of preoperative image is limited, we adopt the bidirectional hybrid mixture models (HMMs) to represent the point set pairs and formulate the probabilistic registration network. In the proposed novel correspondence network where a dual-path cross-attention mechanism is adopted for feature fusion and a clustering mechanism is leveraged for calculating point-to-mixture correspondences. Furthermore, the bidirectional registration mechanism is leveraged to compute the transformation based on estimated correspondences. Third, we have extensively validated the proposed approach on various datasets and bone phantoms. Our experiments on 1399 human femur and 1301 hip models demonstrate that our method achieves state-of-the-art performance across overlap rates from 15% to 35% and at various point counts (i.e., 25, 50, and 100 points) under conditions with less than 50% overlap. Additionally, real phantom experiments on femur and hip models validate the method’s performance in simulated surgical scenarios. Experiments on ModelNet40 further confirmed our method’s effectiveness and generalizability.

In this paper, we propose a novel unsupervised intraoperative liver deformation correction method, called Learning Coherent point drift Network (LCNet), for image-guided liver surgery (IGLS). We first estimate the correspondences between the preoperative and intraoperative point sets in the optimal transport (OT) module by leveraging both original points and extracted features. Afterwards, we compute the point-wise displacement vector by solving the involved matrix equation in the Transformation module, where the point localisation noise is explicitly considered and modeled. Additionally, we present three variants of the proposed approach, i.e., LCNet, LCNet-ED and LCNet-WD, where better registration performances of LCNet against the other two demonstrate the superiority of the utilised Chamfer loss. We have extensively evaluated LCNet on the MedShapeNet dataset consisting of 615 different liver shapes of real patients, and the 3Dircadb dataset comprising 20 liver models of real patients. Extensive experimental results under different deformation and noise magnitudes demonstrate that LCNet outperforms existing state-of-the-art registration algorithms and holds significant application potential in IGLS. For example, when the overlapping ratio between the preoperative and intraoperative point sets is 25%, the deformation magnitude is 8 mm, the maximum point localization noise magnitude is 2 mm and the rotation angle lies in the range of [−45°, 45°], LCNet achieves a root-mean-square error (RMSE) value being 3.21 mm on MedShapeNet dataset, significantly outperforming those of Lepard and RoITr being 5.41 mm (p < 0.001) and 4.90 mm (p < 0.001) respectively.

In this paper, we present a novel curve-to-surface registration method, termed Bi-directional Hybrid Mixture Model Registration based on Dual-constrained Tangent and Normal Vectors (BiHMM-DTN), where two different tangent vectors at the intraoperative point are simultaneously used with the normal vector at the corresponding preoperative point to construct the geometric constraints. While hybrid registration models incorporating tangent and normal vectors (HMM-TN) demonstrate success, their geometric constraints prove inadequate or inappropriate for sparse intraoperative point sets, frequently yielding suboptimal optimization outcomes. By critically revisiting the geometric constraints of HMM-TN, we propose a dual-constraints-based hybrid mixture model registration framework with enhanced intraoperative point set acquisition protocols. To deal with noise and outliers in preoperative and intraoperative point sets—caused by reconstruction inaccuracies and tracking errors, respectively—our approach employs a bi-directional registration mechanism for curve-to-surface registration. We provide rigorous proofs validating the geometric completeness of the dual constraints within this mechanism. The BiHMM-DTN framework is formulated as a maximum likelihood estimation (MLE) problem and optimized using an expectation-maximization (EM) algorithm. Furthermore, to enhance convergence stability and accelerate optimization, the rotation matrix is updated iteratively through successive incremental steps. Extensive experiments on human femur and hip models demonstrate that our method outperforms state-of-the-art approaches, including both traditional optimization and deep learning methods, under various noise and outlier conditions. Furthermore, real-world phantom experiments highlight the potential clinical value of our method for surgical navigation applications. The codes and data are available at https://github.com/sam-zyzhang/BiHMM-DTN.git.

Soft grippers conform to the shape and surface properties of the objects to be grasped, effectively avoiding damage to soft and fragile items. Despite the variety of existing soft gripper designs, their structures lack sufficient flexibility for effectively grasping slender objects or operating in narrow spaces. To address these challenges, we propose a soft gripper with single-finger grasping capabilities, inspired by the structure of crab claws. The structural design and the fabrication method of the gripper are introduced, and the analytical bending model is derived. Experiments are conducted under typical operating conditions to validate the model, and the results indicate that the measured data are in good accordance with the predicted responses. Furthermore, a series of grasping experiments are carried out to test the single-finger grasping capabilities of the proposed soft gripper. The results indicate that the proposed soft gripper can efficiently and stably grasp slender or irregular objects with a single finger. In particular, it demonstrates suitability for operations in narrow spaces and shows potential for handling complex tasks. This innovative design effectively reduces the complexity of the system, while exhibiting promising capabilities in grasping slender or irregular objects and operating within restricted spaces.

3D point cloud registration is an essential problem in computer vision, robotics, surgical navigation and augmented reality. Accurate registration of partially overlapped intraoperative point clouds (e.g., femoral reconstruction) remains critical yet challenging in orthopedic navigation due to incomplete overlap and dynamic noise. In this study, we propose a partial-to-partial point cloud registration framework based on directional spatial consistency. First, we extract overlapped areas from partially overlapping point clouds and leverage the point registration graph matching module to calculate the hard point matching matrix. Second, we sample nodes from the source point cloud and generate translation-invariant edge vectors (direction/scale-preserving) via their k-nearest neighbors, guided by predicted point correspondences. This bypasses translation ambiguities by encoding spatial consistency through edges, reducing pose estimation to 3DoF alignment (rotation). The loss explicitly couples point-level matches with edge-level geometric constraints for dual optimization. Building upon this framework, we extract reliable overlapping edge representations and prune their similarity matrix by thresholding low-confidence scores, effectively suppressing spurious matches. The proposed edge-aware matching mechanism further exploits the translation invariance of local structures to refine point correspondences with enhanced accuracy. Finally, we introduce a bidirectional registration mechanism to reinforce optimization stability, achieving state-of-the-art performance across benchmarks. Extensive experiments on ModelNet40, ShapeNet, and MedShapeNet validate our method under diverse scenarios: partial-to-partial, unseen categories, partial-to-full, and cross-dataset generalization, surpassing existing methods in registration accuracy. The codes are available at https://github.com/pidan0824/DSCGM.

In this paper, we introduce a novel overlap-based bidirectional point set registration approach, i.e., Overlap-guided Bidirectional Hybrid Mixture Registration (OBHMR), which incorporates geometric information (i.e., normal vectors) in both the correspondence and transformation stages and formulates the optimization objective of registration in a bidirectional manner. More importantly, to address the issue of partial-to-full registration, OBHMR utilises the predicted point-wise overlap score using networks to formulate the overlap-guided Hybrid Mixture Model consisting of the Gaussian Mixture Model (GMM) and Fisher Mixture Model (FMM). OBHMR contains four components: (1) the overlap-guided correspondence network that estimates the correspondence probabilities and calculates the point-wise overlap score; (2) the learning posterior module that estimates the overlap-guided HMM parameters; (3) the transformation module that computes the rigid transformation by formulating the optimisation objective in a bidirectional registration way, given correspondences and overlap-guided HMM parameters. Experiments using 1457 human femur and 1301 human hip models demonstrate significant improvements in partial-to-full registration performance (p < 0.01) under different overlapping ratios, compared to state-of-the-art registration approaches. Furthermore, individual contributions of three modules (i.e., additional normal vectors, overlap score estimation module and the bidirectional mechanism) in OBHMR have been validated in ablation studies. The results demonstrate OBHMR’s capability of tackling the challenging partial-to-full registration problems in computer-assisted orthopedic surgery. The codes are available at https://github.com/Dxinz/DeepOBHMR.

In this paper, we introduce a novel normal-assisted learning-based rigid registration approach, i.e., Deep Bi-directional Hybrid Mixture Registration (DeepBHMR). Our approach utilises helpful normal vectors explicitly in both correspondence and transformation stages and formulates the optimization objective of registration in a bi-directional way that considers noise in both point sets. DeepBHMR consists of three modules: (1) the correspondence network that estimates the correspondence probability relating points within one generalized point set (i.e., positional and normal vectors) with components of Hybrid Mixture Models (HMMs) representing the other generalized point set; (2) the posterior module that computes HMMs parameters; (3) the transformation module that computes the rotation matrix and the translation vector given the estimated generalized-point to hybrid-distribution correspondences and HMMs parameters. DeepBHMR has been validated on 291 human femur and 260 hip models, and extensive experimental results demonstrate that DeepBHMR outperforms the state-of-the-art registration methods (p-value < 0.01). In the circumstance of femur bones, the mean rotation and translation error values are around 1° (i.e., 1.01°) and less than 1 mm (i.e., 0.36mm), respectively. Furthermore, even under the large transformation (i.e., in the range of [0,180]° and [0, 100] mm), the mean RMSE values being 3.05 mm is still satisfactory. Additionally, the results demonstrate the DeepBHMR’s favorable generalizability from femur shapes to hip shapes. We have carefully validated the significant benefits of incorporating normal vectors and the bidirectional mechanism. DeepBHMR can successfully handle the challenging scenario of large transformation and partial registration. The codes are available at https://github.com/zzyrobot/DeepBHMR.git.

In this paper, we introduce Bidirectional Non-Overlapping Filtering Network (Bi-NOFNet), which registers the partial intraoperative point set with full preoperative point set for computer-assisted interventions (CAI). Our contributions are three-folds. First, Bi-NOFNet adopts customised feature extractor to extract distinctive features from both point sets, with which the per-point overlap mask is predicted and the overlapping region is segmented for the preoperative point set. Furthermore, we propose two methods to filter out the non-overlapping regions, at feature-level (i.e., Bi-NOFNet(Feature)) and point-level (i.e., Bi-NOFNet (Point)). For these two methods, we develop supervised registration strategy where the ground-truth overlap mask and displacement vectors are employed, and weakly-supervised registration strategies where only the ground-truth overlap mask is available. Additionally, to fully utilise the information in both space, we propose a bidirectional registration mechanism, which predicts the displacement vectors associated with the intraoperative point set (i.e., the forward way) and those warpping the preoperative point set (i.e., the backward way). Experiments have been conducted on the proposed DeformMedShapeNet dataset that contains 615 different liver shapes. Extensive results demonstrate that Bi-NOFNet performs well for partial-to-full registration tasks under various scenarios of noise, overlap ratios and deformation levels, outperforming existing non-rigid registration approaches.

Convolution is a fundamental operation in the 3D backbone. However, under certain conditions, the feature extraction ability of traditional convolution methods may be weakened. In this paper, we introduce a new convolution method based on $\ell_p$-norm. For theoretical support, we prove the universal approximation theorem for $\ell_p$-norm based convolution, and analyze the robustness and feasibility of $\ell_p$-norms in 3D point cloud tasks. Concretely, $\ell_{\infty}$-norm based convolution is prone to feature loss. $\ell_2$-norm based convolution is essentially a linear transformation of the traditional convolution. $\ell_1$-norm based convolution is an economical and effective feature extractor. We propose customized optimization strategies to accelerate the training process of $\ell_1$-norm based Nets and enhance the performance. Besides, a theoretical guarantee is given for the convergence by \textit{regret} argument. We apply our methods to classic networks and conduct related experiments. Experimental results indicate that our approach exhibits competitive performance with traditional CNNs, with lower energy consumption and instruction latency.

In medical robotics and image-guided surgery (IGS), registration is needed in order to align together the coordinate frames of robots, medical imaging modalities, surgical tools, and patients. Existing registration algorithms often assume one point set to be a noise-free model while the other to contain noise and outliers. However, in real scenarios, noise and outliers can exist in both point sets to be registered. To eliminate the above-mentioned challenge, in this paper, we formally formulate the Bi-directional Generalised Rigid Point Set Registration (Bi-GRPSR) problem where normal vectors are adopted, bi-directional probability density function (PDFs) and Hybrid Mixture Models (HMMs) are constructed to derive the objective function. Bi-GRPSR considering anisotropic positional noise is thus cast as a maximum likelihood estimation (MLE) problem, which is solved by the proposed Bi-directional Generalised Anisotropic Coherent Point Drift (Bi-AGCPD) where spatially nearby points are considered to move coherently and iterative expectation maximization (EM) steps are involved. Experimental results on two human bone point sets, under different settings of noise, outliers, and overlapping ratios, validate the effectiveness and improvements of Bi-AGCPD over existing probabilistic and learning-based methods.Note to Practitioners—This paper presents a novel rigid point set registration method that explicitly takes the anisotropic noise in both point sets into account. The proposed framework first formulates the probability density functions of generalised points in a bi-directional way, with which the bi-directional hybrid mixture model is built. The resulting objective function is minimised with the expectation maximisation technique. The algorithms are essential for real-world applications in that noise usually exists in both spaces to be registered and is generally anisotropic. The proposed method has demonstrated promising results on two human femur bone models, which indicates the great potential for it to be readily applied to related applications, especially medical scenarios, given that the two point sets are coarsely aligned. Future work will extend the presented method into scenarios of global registration.

In medical robotics and image-guided surgery (IGS), registration is needed in order to align together the coordinate frames of robots, medical imaging modalities, surgical tools, and patients. Existing registration algorithms often assume one point set to be a noise-free model while the other to contain noise and outliers. However, in real scenarios, noise and outliers can exist in both point sets to be registered. To eliminate the above-mentioned challenge, in this paper, we formally formulate the Bi-directional Generalised Rigid Point Set Registration (Bi-GRPSR) problem where normal vectors are adopted, bi-directional probability density function (PDFs) and Hybrid Mixture Models (HMMs) are constructed to derive the objective function. Bi-GRPSR considering anisotropic positional noise is thus cast as a maximum likelihood estimation (MLE) problem, which is solved by the proposed Bi-directional Generalised Anisotropic Coherent Point Drift (Bi-AGCPD) where spatially nearby points are considered to move coherently and iterative expectation maximization (EM) steps are involved. Experimental results on two human bone point sets, under different settings of noise, outliers, and overlapping ratios, validate the effectiveness and improvements of Bi-AGCPD over existing probabilistic and learning-based methods.

This paper presents a novel augmented reality (AR)-assisted orthopedic surgical robotic system based on Head-Mounted Display (HMD) devices. The proposed system can overlay the preoperative plans over the patient's anatomy and provide useful guidance for surgeons during interventions, with integrated calibration and registration components. A novel bi-directional generalised point set registration algorithm that utilises robust features is developed to accurately align the pre-operative CT and intra-operative patient spaces, which has been demonstrated to outperform existing registration methods. The efficacy of the system is both qualitatively and quantitatively assessed with an in vitro study simulating a total knee arthroplasty (TKA) procedure. The experimental results showed that 1) the system can successfully align the preoperative and intraoperative spaces, with the mean target registration error (TRE) being 2.7771 mm; 2) the models can be properly overlaid to the physical scenarios with the mean AR visualization accuracy being 6.9726 mm.

The prowess that makes few-shot learning desirable in medical image analysis is the efficient use of the support image data, which are labelled to classify or segment new classes, a task that otherwise requires substantially more training images and expert annotations. This work describes a fully 3D prototypical few-shot segmentation algorithm, such that the trained networks can be effectively adapted to clinically interesting structures that are absent in training, using only a few labelled images from a different institute. First, to compensate for the widely recognised spatial variability between institutions in episodic adaptation of novel classes, a novel spatial registration mechanism is integrated into prototypical learning, consisting of a segmentation head and an spatial alignment module. Second, to assist the training with observed imperfect alignment, support mask conditioning module is proposed to further utilise the annotation available from the support images. Extensive experiments are presented in an application of segmenting eight anatomical structures important for interventional planning, using a data set of 589 pelvic T2-weighted MR images, acquired at seven institutes. The results demonstrate the efficacy in each of the 3D formulation, the spatial registration, and the support mask conditioning, all of which made positive contributions independently or collectively. Compared with the previously proposed 2D alternatives, the few-shot segmentation performance was improved with statistical significance, regardless whether the support data come from the same or different institutes.

Registration is highly demanded in many real-world scenarios such as robotics and automation. Registration is challenging partly due to the fact that the acquired data is usually noisy and has many outliers. In addition, in many practical applications, one point set (PS) usually only covers a partial region of the other PS. Thus, most existing registration algorithms cannot guarantee theoretical convergence. This article presents a novel, robust, and accurate three-dimensional (3D) rigid point set registration (PSR) method, which is achieved by generalizing the state-of-the-art (SOTA) Bayesian coherent point drift (BCPD) theory to the scenario that high-dimensional point sets (PSs) are aligned and the anisotropic positional noise is considered. The high-dimensional point sets typically consist of the positional vectors and normal vectors. On one hand, with the normal vectors, the proposed method is more robust to noise and outliers, and the point correspondences can be found more accurately. On the other hand, incorporating the registration into the BCPD framework will guarantee the algorithm’s theoretical convergence. Our contributions in this article are three folds. First, the problem of rigidly aligning two general PSs with normal vectors is incorporated into a variational Bayesian inference framework, which is solved by generalizing the BCPD approach while the anisotropic positional noise is considered. Second, the updated parameters during the algorithm’s iterations are given in closed-form or with iterative solutions. Third, extensive experiments have been done to validate the proposed approach and its significant improvements over the BCPD. Note to Practitioners—This paper was motivated by the problem of 3D rigid PSR for computer-assisted surgery (CAS), especially in orthopedic applications. The proposed algorithm is also suitable for other scenarios where the initial coarse registration is conducted. The traditional registration methods are susceptible to noise (especially anisotropic noise), outliers, and incomplete partial data. This paper generalizes the recently proposed BCPD method to the six-dimensional scenario where anisotropic positional noise is considered and normal vectors are incorporated. The proposed noise model is decomposed into three parts to be solved alternately: the membership probability of mixture distributions, the soft correspondence estimation, and the model parameters (i.e., the rotation matrix, translation vector, the covariance matrix with the anisotropic positional error, and the concentration parameter with the estimation of the normal vectors). Especially, the convergence is guaranteed at the theoretical level using the variational inference theory. The experimental results demonstrate the superiority of our algorithm on registration accuracy, convergence speed, and robustness to noise, outliers, and partial data.

This paper presents a novel, robust, and accurate three-dimensional (3D) rigid point set registration (PSR) method, which is achieved by generalizing the state-of-the-art (SOTA) Bayesian coherent point drift (BCPD) theory to the scenario that high-dimensional point sets(PSs) are aligned and that the anisotropic positional noise is considered. Our contributions in this paper are three folds. First, the problem of rigidly aligning two general point sets (PSs) with normal vectors is incorporated into a variational Bayesian inference framework, which is solved by generalizing the BCPD approach while the anisotropic positional noise is considered. Second, the updated parameters during the algorithm's iterations are given in closed-form or iterative solutions. Third, extensive experiments have been done to validate the proposed approach and its significant improvements over the BCPD.

In medical image analysis (MIA) and computer-assisted surgery (CAS), aligning two multiple point sets (PSs) together is an essential but also a challenging problem. For example, rigidly aligning multiple point sets into one common coordinate frame is a prerequisite for statistical shape modelling (SSM). Accurately aligning the pre-operative space with the intra-operative space in CAS is very crucial to successful interventions. In this article, we formally formulate the multiple generalized point set registration problem (MGPSR) in a probabilistic manner, where both the positional and the normal vectors are used. The six-dimensional vectors consisting of both positional and normal vectors are called as generalized points. In the formulated model, all the generalized PSs to be registered are considered to be the realizations of underlying unknown hybrid mixture models (HMMs). By assuming the independence of the positional and orientational vectors (i.e., the normal vectors), the probability density function (PDF) of an observed generalized point is computed as the product of Gaussian and Fisher distributions. Furthermore, to consider the anisotropic noise in surgical navigation, the positional error is assumed to obey a multi-variate Gaussian distribution. Finally, registering PSs is formulated as a maximum likelihood (ML) problem, and solved under the expectation maximization (EM) technique. By using more enriched information (i.e., the normal vectors), our algorithm is more robust to outliers. By treating all PSs equally, our algorithm does not bias towards any PS. To validate the proposed approach, extensive experiments have been conducted on surface points extracted from CT images of (i) a human femur bone model; (ii) a human pelvis bone model. Results demonstrate our algorithm’s high accuracy, robustness to noise and outliers. Note to Practitioners—This paper was motivated by solving the problem of registering two or more PSs. Most existing registration approaches use only the positional information associated with each point, and thus lacks robustness to noise and outliers. Three significant improvements are brought by our proposed approach. First, the normal vectors that can be extracted from the point sets are utilized in the registration. Second, the positional error distribution is assumed to be anisotropic and inhomogeneous. Third, all the PSs to be registered are treated equally that means no PS is considered as the model one. The registration problem is cast into a maximum likelihood (ML) problem and solved under the expectation maximization (EM) framework. We have demonstrated through extensive experiments that the proposed registration approach achieves significantly improved accuracy, robustness to noise and outliers. The algorithm is particularly suitable for biomedical applications involving the registration procedures, such as image-guided surgery.

Point set registration (PSR) is an essential problem in surgical navigation and computer-assisted surgery (CAS). In CAS, PSR can be used to map the intraoperative surgical space with the preoperative volumetric image space. The performances of PSR in real-world surgical scenarios are sensitive to noise and outliers. This article proposes a novel point set registration approach where the additional features (i.e., the normal vectors) extracted from the point sets are utilized and the convergence of the algorithm is guaranteed from the theoretical perspective. More specifically, we formulate the PSR with normal vectors by generalizing the Bayesian coherent point drift (BCPD) into the 6-D scenario. The proposed algorithm is more accurate and robust to noise and outliers, and the theoretical convergence of the proposed approach is guaranteed. Our contributions of this article are summarized as follows. 1) The PSR problem with normal vectors is formally formulated through generalizing the BCPD approach. 2) The formulas for updating the parameters during the algorithm’s iterations are given in closed forms. 3) Extensive experiments have been done to verify the proposed approach and specifically its significant improvements over the BCPD has been validated.

Point set registration (PSR) is an essential problem in communities of computer vision, medical robotics and biomedical engineering. This paper is motivated by considering the anisotropic characteristics of the error values in estimating both the positional and orientational vectors from the PSs to be registered. To do this, the multi-variate Gaussian and Kent distributions are utilized to model the positional and orientational uncertainties, respectively. Our contributions of this paper are three-folds: (i) the PSR problem using normal vectors is formulated as a maximum likelihood estimation (MLE) problem, where the anisotropic characteristics in both positional and normal vectors are considered; (ii) the matrix forms of the objective function and its associated gradients with respect to the desired parameters are provided, which can facilitate the computational process; (iii) two approaches of computing the normalizing constant in the Kent distribution are compared. We verify our proposed registration method on various PSs (representing pelvis and femur bones) in computer- assisted orthopedic surgery (CAOS). Extensive experimental results demonstrate that our method outperforms the state- of-the-art methods in terms of the registration accuracy and the robustness.

Point set registration (PSR) is an essential problem in surgical navigation and image-guided surgery (IGS). It can help align the pre-operative volumetric images with the intra-operative surgical space. The performances of PSR are susceptible to noise and outliers, which are the cases in real-world surgical scenarios. In this paper, we provide a novel point set registration method that utilizes the features extracted from the PSs and can guarantee the convergence of the algorithm simultaneously. More specifically, we formulate the PSR with normal vectors by generalizing the bayesian coherent point drift (BCPD) into the six-dimension scenario. Our contributions can be summarized as follows. (1) The PSR problem with normal vectors is formulated by generalizing the Bayesian coherent point drift (BCPD) approach; (2) The updated parameters during the algorithm's iterations are given in closed-forms; (3) Extensive experiments have been done to verify the proposed approach and its significant improvements over the BCPD has been validated. We have validated our proposed registration approach on both the human femur model. Results demonstrate that our proposed method outperforms the state-of-the-art registration methods and the convergence is guaranteed at the same time.

Nonrigid point set (PS) registration is an outstanding and fundamental problem in the fields of robotics, computer vision, medical image analysis, and image-guided surgery (IGS). The aim of a nonrigid registration problem is to align together two point sets where one has been deformed. The assumption of isotropic localization error is shared in the previous nonrigid registration algorithms. In this article, we have derived and presented a novel nonrigid registration algorithm, where the position localization error (PLE) is generalized to be anisotropic, which means that the error distribution is not the same in different spatial directions. The motivation of considering the anisotropic characteristic is that the PLE is actually different in three spatial directions in real applications of registrations, such as IGS. Mathematically, the difficulty in dealing with the anisotropic error case comes from the change from a standard deviation that is a scalar to a covariance matrix. The formulas for updating the parameters in both expectation and maximization steps are derived. More specifically, in the expectation step, we compute the posterior probabilities that represent the correspondences between points in two PSs. In the maximization step, given the current posteriors, the covariance matrix of the PLE and the nonrigid transformation are updated. To further speed up the proposed algorithm, the low-rank approximation variation of our method is also presented. We have demonstrated through experiments on both general and medical data sets (corrupted with noise) that the proposed algorithm outperforms the state-of-the-art ones in terms of registration accuracy and robustness to noise. More specifically, all the experimental results have passed the statistical tests at the 5% significance level.Note to Practitioners—This article was motivated by solving the problem of nonrigidly registering two point sets where one has been deformed and corrupted with anisotropic noise. Most existing registration methods generally assume the positional error to be the same in all directions, which in fact is not the case in real scenarios. This article presents a new robust method that assumes the positional error to be anisotropic, which is the case in point sets coming from the stereo reconstruction. The nonrigid registration problem is formulated as a maximum-likelihood (ML) problem and solved with the expectation–maximization (EM) technique. We have demonstrated through extensive experiments on both general and medical data sets that the proposed registration algorithm achieves significantly improved accuracy, robustness to noise, and outliers compared with state-of-the-art algorithms. The algorithm is particularly suitable for biomedical applications involving the registration, such as medical imaging and image-guided surgery (IGS).

Registering two point sets (PSs) is an essential problem in medical robotics and computer-assisted surgery (CAS). As one typical example, in computer-assisted orthopedic surgery (CAOS), the preoperative scan has to be aligned with the intraoperative scan accurately. In this article, we first formally formulate the generalized PS registration problem in a probabilistic manner. Especially, not only positional but also orientational information is incorporated into the registration. Notably, the positional error is assumed to obey a multivariate Gaussian distribution to accommodate the anisotropic noise. The expectation–maximization (EM) framework is utilized to solve the maximum likelihood (ML) problem. In the E-step, the correspondence probabilities between points in two generalized PSs are computed. In the M-step, the constrained optimization problem with respect to the rigid transformation matrix is reformulated as an unconstrained one. This is achieved by utilizing the Rodrigues parameterization to represent the rotation matrix. Both extensive simulated and real experiments are conducted to validate the proposed algorithm by comparing it with state-of-the-art registration methods.Note to Practitioners—This article was motivated by considering the anisotropic positional uncertainty into the rigid point set (PS) registration in the application of preoperative-to-intraoperative registration within image-guided surgery. We provide iterative solutions that compute the rotation and translation vector that aligns two 3-D PSs. The correspondences between points in two PSs are not known and regarded as hidden variables in the optimization process. Expectation–maximization technique is utilized to solve the maximum likelihood problem. We have demonstrated through experiments that our proposed approach can achieve lower registration error values than the compared state-of-the-art registration methods on various data sets. The readers should note that the proposed method is particularly suitable for cases that anisotropic noise is involved.

Registration is an essential problem in image-guided surgery (IGS) since it brings different involved coordinate frames together. Nonrigid or deformable registration still faces many challenges, such as two point sets (PSs) are partially overlapped. To tackle the challenges in the nonrigid registration, we introduce a new two-step point-based registration pipeline that includes two steps. In the first step, the rigid transformation between the two spaces is recovered where the orientation vectors are adopted. In the second step, built on the nonrigid coherent point drift (CPD) approach, the anisotropic positional noise is also assumed. Registration results on the human liver verify the proposed approach' great improvements over the other methods. First, the rotation and translation are recovered with smaller error values than the existing methods. Second, our registration method's performance is much more robust to the partial overlapping between two PSs. Third, the two-step registration framework achieves the best performances in most test cases when there is a localization error in acquiring the intraoperative data. Note to Practitioners-A novel registration approach is presented for image-guided liver surgery (LGLS). Compared with existing nonrigid registration methods, two significant changes (or improvements) exist in the proposed registration framework: 1) the normal vectors are extracted and utilized in the rigid registration step and 2) the anisotropic positional uncertainties are considered. In both steps, the registration problems are formulated as a maximum likelihood (ML) problems and dealt with the expectation-maximization (EM) technique. In both steps, the matrix form of the updated positional covariance is provided and can speed up the computational process. The readers are reminded that with extra information and a more general positional error assumption, our approach demonstrates improved performances in the case of partial-to-full alignment.

This paper presents a robust and accurate approach for the rigid registration of pre-operative and intraoperative point sets in image-guided surgery (IGS). Three challenges are identified in the pre-to-intraoperative registration: the intra-operative 3D data (usually forms a 3D curve in space) (1) is often contaminated with noise and outliers; (2) usually only covers a partial region of the whole pre-operative model; (3) is usually sparse. To tackle those challenges, we utilize the tangent vectors extracted from the sparse intraoperative data points and the normal vectors extracted from the pre-operative model points. Our first contribution is to formulate a novel probabilistic distribution of the error between a pair of corresponding tangent and normal vectors. The second contribution is, based on the novel distribution, we formulate the registration of two multi-dimensional (6D) point sets as a maximum likelihood (ML) problem and solve it under the expectation maximization (EM) framework. Our last contribution is, in order to facilitate the computation process, the derivatives of the objective function with respect to desired parameters are presented. We conduct extensive experiments to demonstrate that our approach outperforms the state-of-the-art methods. Importantly, in the context of anteriro cruciate ligament (ACL) reconstruction, our method can achieve as low as 0.6795 mm mean target registration error (TRE) value with considerable noises and very limited overlapping ratios.

In a paired-point rigid registration, target registration error (TRE) is deemed to be the most important quality metric. TRE usually cannot be directly measured, and thus many TRE estimation algorithms have been proposed. However, target localization errors (TLEs) in two spaces are not considered in the definition of TRE. In this paper, we propose a new type of evaluation metric that is referred to as total TRE (TTRE) at a given target point with TLE incorporated. Statistics including mean, root mean square (rms), and covariance matrix of TTRE are derived without making any assumption of the TLE magnitude. TTRE and fiducial registration error (FRE) are proved to be uncorrelated when an ideal weighting scheme is adopted in solving the registration problem. The proposed error model is validated through extensive experiments. In the first experiment with random fiducials and targets, in 90% of the test cases, there shows no difference between the predicted and simulated TTRE statistics when six fiducials are used. In the second experiment of deep-brain stimulation surgery, the mean value of CC(TTRE,FRE) being 8.9246 × 10-4± 0.0389 was observed, which indicates that TTRE and FRE are uncorrelated. In the third experiment of surgical tool-tip tracking, the mean and standard deviation of percentage differences between predicted and simulated TTRE rms values are 2.22% ± 0.77% for the planar tool and 2.62% ± 0.59% for the textral tool. In summary, our proposed algorithm can well model the TTRE metric.

This paper introduces a robust generalized point cloud (PC) registration method that utilizes not only the positional but also the orientation information associated with each point. The proposed method solves the rigid PC registration problem in a probabilistic manner, which casts the problem into a maximum likelihood (ML) framework. A hybrid mixture model (HMM) is utilized to represent one generalized PC. In the HMM, a von-Mises-Fisher mixture model (FMM) is adopted to model the orientational uncertainty, while a Gaussian mixture model (GMM) is used to represent the positional uncertainty. An expectation-maximization (EM) algorithm is adopted to solve the optimization problem in an iterative manner to find the optimal rotation matrix and the translation vector between two generalized PCs. In both expectation step (E step) and maximization step (M step), orientational information is utilized, which can potentially improve the algorithm's robustness to noise and outliers. In the E step, the posterior probabilities that represent the degree of point correspondences in two PCs are computed. In the M step, an efficient closed-form solution to a rigid transformation matrix is developed. E and M steps will iterate until certain convergence criteria are satisfied. Extensive experiments under different noise levels and outlier ratios have been carried out on a data set of femur bone computed tomography images. Experimental results show that the proposed method outperforms the state-of-the-art ones in terms of accuracy, robustness, and convergence speed significantly. Note to Practitioners-This paper was motivated by solving the problem of registering two PCs. Most existing approaches generally use only the positional information associated with each point and thus lack robustness to noise and outliers. This paper suggests a new robust method that also adopts the normal vectors associated with each point. The registration problem is cast into a maximum likelihood (ML) problem and solved under the expectation-maximization (EM) framework. Closed-form solutions for estimating parameters in both expectation and maximization steps are provided in this paper. We have demonstrated through extensive experiments that the proposed registration algorithm achieves improved accuracy, robustness to noise and outliers, and faster convergence speed.

Aligning different views or representations of anatomy is an essential task in both medical imaging computing (MIC) and computer-assisted interventions' (CAIs') communities. Motivated by simultaneously registering multiple point sets (PSs) and further improving the algorithm's robustness to outliers and noise, in this paper, we propose a novel probabilistic approach to jointly register multiple generalized PSs. A generalized PS includes high-dimensional points consisting of both positional vectors and orientational information (or normal vectors). Hybrid mixture models (HMMs) combining Gaussian and von Mises-Fisher (VMF) distributions are used to model positional and orientational components of the generalized PSs. All generalized PSs are jointly registered using the expectation-maximization (EM) technique. In the E-step, the posterior probabilities representing point correspondence confidences are computed. In the M-step, the rigid transformation matrices, positional variances, and orientational concentration parameters are updated for each generalized PS. E and M steps will iterate until some termination condition is satisfied. We validate our algorithm using the surface points extracted from the human femur CT model. The experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art ones in terms of the accuracy, robustness, as well as convergence speed. In addition, our algorithm is able to recover a better central PS than the state-of-the-art one does in the case of registering multiple PSs. Our algorithm is very suitable for registering complex structures arising in medical imaging. This paper was motivated by solving the problem of registering two or multiple point sets. Most existing approaches generally use only the positional information associated with each point and thus lack robustness to noise and outliers. This paper suggests a new robust method that also adopts the normal vectors associated with each point. The registration problem is cast into a maximum-likelihood (ML) problem and solved under the expectation-maximization (EM) framework. Closed-form solutions to estimating parameters in both expectation and maximization steps are provided in this paper. We have demonstrated through extensive experiments that the proposed registration algorithm achieves improved accuracy, robustness to noise and outliers, and faster convergence speed.

Point set registration (PSR) is an important problem in computer vision, robotics and biomedical engineering communities. Usually, only positional information at each point is adopted in a registration. In this paper, the orientational vector (or normal vector) associated with each point is also utilized. Generalized point set registration is formulated and solved under the Expectation-Maximization (EM) framework. In the E-step, the posterior probabilities representing the correspondence probabilities are computed. In the Mstep, rigid transformation parameters including the rotation matrix, the translation vector are updated. The proposed algorithm stops when it converges to the optimal solution or a maximum number of iterations is achieved. The observed position set and normal vector set are assumed to follow Gaussian Mixture Models (GMMs) and Fisher distribution Mixture Models (FMMs), respectively. To further improve our algorithm's robustness, the hybrid mixture models (HMMs) are assumed to be inhomogeneous. Experimental results on the surface points extracted from a human femur' CT model show that our algorithm can achieve lower registration error, is more robust to noise and outliers than the state-of-the-art registration methods.

Non-rigid point set registration (PSR) is an outstanding and fundamental problem in fields of robotics, computer vision, medical image analysis and imageguided surgery. The aim of a non-rigid registration problem is to align together two point sets that have been deformed. We have derived and presented a novel registration algorithm that non-rigidly registers two point sets together. The assumption of isotropic localization error is shared in the previous non-rigid registration algorithms. In this paper, the position localization error is generalized to the anisotropic cases, which means that the error distribution is not the same in different spatial directions. The motivation of considering the anisotropic characteristic is that the point localization error is actually different in three spatial directions in real applications. Mathematically, the difficulty in dealing with the anisotropic error case comes from the change from a standard deviation that is a scalar to a covariance matrix. The formulas for updating the parameters in both expectation and maximization steps are derived. In the expectation step, we compute the posterior probabilities that represent the correspondences between points in two PSs. In the maximization step, given the current posteriors, the covariance matrix of the position localization error and the non-rigid transformation are updated. To facilitate the proposed algorithm, the low-rank approximation variation of our method is also presented. We have demonstrated through experiments that the proposed algorithm outperforms the state-of-the-art ones in terms of registration and accuracy and robustness to noise. More specifically, most of the experimental results have passed the statistical tests at the 5% significance level.

Alignment of multiple point sets is an essential problem in medical imaging and computer-assisted surgery. For example, aligning multiple point sets into one common coordinate frame is a prerequisite for statistical shape modelling (SSM). In this paper, we first formally formulate the multiple generalized point cloud registration problem in a probabilistic manner. Not only positional but also the orientational information is utilized in the registration. All the observed generalized point sets to be registered are considered to be realizations of underlyinng unknown hybrid mixture models (HMMs). By (i) utilizing more enriched information, i.e. orientational information or normal vectors (ii) treating all point sets equally, our registration algorithm is more robust to outliers and does not bias towards any point set. Assuming that the positional and orientational data are co-independent, the probability density function (PDF) of an observed hybrid point is the multiplication of Gaussian and Fisher distributions. Notably, the positional error vector is assumed to obey a multivariate Gaussian distribution to accommodate anisotropic noise. Expectation maxmization (EM) framework is utilized to jointly estimate the parameters. In the E-step, the posteriors between points and underlying mixture model components are computed. In the M-step, the constrained optimization problem of the rigid transformation matrix is re-formulated as an unconstrained one using the Rodrigues Formula of a rotation matrix. Extensive experiments are conducted on CT data of a femur bone model to compare the proposed algorithm with the state-of-the-art registration methods. The experimental results demonstrate the algorithm's better accuracy, robustness to noise and outliers and faster convergence speed.

This paper introduces a robust point cloud registration method which utilizes not only positional but also the orientation information at each point. The proposed method takes a probabilistic approach which forms the problem as a hybrid mixture model, in which a Von-Mises-Fisher mixture model (FMM) is adopted to model the orientation part and a gaussian mixture model (GMM) is used to represent the position part. When two point clouds are optimally registered, the correspondence is the maximum of the posterior probability of the overall mixture model. Expectation-Maximization (EM) algorithm has been adopted to solve the optimization problem in an iterative manner to find the optimal rotation and translation between two point clouds. Extensive experiments under different noise levels and different outlier ratios have been carried out on a dataset of the femur CT images. Comparison results show that the proposed method outperforms the state-of-the-art methods under most of the experimental conditions, which indicates the validity of our method.

Alignment of two point clouds is an essential problem in medical robotics and computer-assisted surgery. In this paper, we first formally formulate the generalized point cloud registration problem in a probabilistic manner. Specifically, not only positional but also the orientational information are incorporated into registration. Notably, the positional error is assumed to obey a multivariate Gaussian distribution to accommodate anisotropic cases. Expectation conditional maximization framework is utilized to solve the problem. In E-step, the correspondence probabilities between points in two generalized point clouds are computed. In M -step, the constrained optimization problem with respect to the transformation matrix is re-formulated as an unconstrained one. Extensive experiments are conducted to compare the proposed algorithm with the state-of-the-art registration methods. The experimental results demonstrate the algorithm's robustness to noise and outliers, fast convergence speed.

Target registration error (TRE) is widely adopted to evaluate the accuracy of a paired-point rigid registration (PPRR). However, TRE is defined in such a way that target localization error (TLE) is not considered. In this paper, we first propose a new type of error that is referred to as total target registration error (TTRE). The statistical model of TTRE is derived that we take the TLE in two spaces to be registered into consideration. Results in the first simulation show that the developed model can accurately estimate the simulated TTRE root-mean-square (RMS) (RMS percent differences|| <; 1.5% ± 2%) in all test cases. When all elements of diagonal FLE and TLE covariance matrices are independently generated from a uniform distribution that spans from 0 to 1mm and the number of fiducials N ≥ 6, the mean and covariance matrix of TTRE are well modelled. We have also theoretically proved and validated through the second simulation that TTRE and fiducial registration error (FRE) are uncorrelated (correlation coefficient (CC) <; 0.1). Finally, TTRE and TRE were found to exhibit a low correlation (0.37 <; CC <; 0.46).