This paper proposes a novel method for removing image feature mismatches in real-time that can handle both rigid and smooth deforming environments. Image distortion, parallax and object deformation may cause the pixel coordinates of feature matches to have non-rigid deformations, which cannot be represented using a single analytical rigid transformation. To solve this problem, we propose an algorithm based on the re-weighting and 1-point RANSAC strategy (R1P-RNSC), which operates under the assumption that a non-rigid deformation can be approximately represented by multiple rigid transformations. R1P-RNSC is fast but suffers from the drawback that local smoothing information cannot be considered, thus limiting its accuracy. To solve this problem, we propose a non-parametric algorithm based on the expectation-maximization algorithm and the dual quaternion-based representation (EMDQ). EMDQ generates dense and smooth deformation fields by interpolating among the feature matches, simultaneously removing mismatches that are inconsistent with the deformation field. It relies on the rigid transformations obtained by R1P-RNSC to improve its accuracy. The experimental results demonstrate that EMDQ has superior accuracy compared to other state-of-the-art mismatch removal methods. The ability to build correspondences for all image pixels using the dense deformation field is another contribution of this paper.

The ability to extend the field of view of laparoscopy images can help the surgeons to obtain a better understanding of the anatomical context. However, due to tissue deformation, complex camera motion and significant three-dimensional (3D) anatomical surface, image pixels may have non-rigid deformation and traditional mosaicking methods cannot work robustly for laparoscopy images in real-time. To solve this problem, a novel two-dimensional (2D) non-rigid simultaneous localization and mapping (SLAM) system is proposed in this paper, which is able to compensate for the deformation of pixels and perform image mosaicking in real-time. The key algorithm of this 2D non-rigid SLAM system is the expectation maximization and dual quaternion (EMDQ) algorithm, which can generate smooth and dense deformation field from sparse and noisy image feature matches in real-time. An uncertainty-based loop closing method has been proposed to reduce the accumulative errors. To achieve real-time performance, both CPU and GPU parallel computation technologies are used for dense mosaicking of all pixels. Experimental results on in vivo and synthetic data demonstrate the feasibility and accuracy of our non-rigid mosaicking method.

We propose an approach to reconstruct dense three-dimensional (3D) model of tissue surface from stereo optical videos in real-time, the basic idea of which is to first extract 3D information from video frames by using stereo matching, and then to mosaic the reconstructed 3D models. To handle the common low-texture regions on tissue surfaces, we propose effective post-processing steps for the local stereo matching method to enlarge the radius of constraint, which include outliers removal, hole filling, and smoothing. Since the tissue models obtained by stereo matching are limited to the field of view of the imaging modality, we propose a model mosaicking method by using a novel feature-based simultaneously localization and mapping (SLAM) method to align the models. Low-texture regions and the varying illumination condition may lead to a large percentage of feature matching outliers. To solve this problem, we propose several algorithms to improve the robustness of the SLAM, which mainly include 1) a histogram voting-based method to roughly select possible inliers from the feature matching results; 2) a novel 1-point RANSAC-based [Formula: see text] algorithm called as DynamicR1PP [Formula: see text] to track the camera motion; and 3) a GPU-based iterative closest points (ICP) and bundle adjustment (BA) method to refine the camera motion estimation results. Experimental results on ex- and in vivo data showed that the reconstructed 3D models have high-resolution texture with an accuracy error of less than 2 mm. Most algorithms are highly parallelized for GPU computation, and the average runtime for processing one key frame is 76.3 ms on stereo images with 960×540 resolution.

The ability to handle outliers is essential for performing the perspective- n-point (P nP) approach in practical applications, but conventional RANSAC+P3P or P4P methods have high time complexities. We propose a fast P nP solution named R1PP nP to handle outliers by utilizing a soft re-weighting mechanism and the 1-point RANSAC scheme. We first present a P nP algorithm, which serves as the core of R1PP nP, for solving the P nP problem in outlier-free situations. The core algorithm is an optimal process minimizing an objective function conducted with a random control point. Then, to reduce the impact of outliers, we propose a reprojection error-based re-weighting method and integrate it into the core algorithm. Finally, we employ the 1-point RANSAC scheme to try different control points. Experiments with synthetic and real-world data demonstrate that R1PP nP is faster than RANSAC+P3P or P4P methods especially when the percentage of outliers is large, and is accurate. Besides, comparisons with outlier-free synthetic data show that R1PP nP is among the most accurate and fast P nP solutions, which usually serve as the final refinement step of RANSAC+P3P or P4P. Compared with REPP nP, which is the state-of-the-art P nP algorithm with an explicit outliers-handling mechanism, R1PP nP is slower but does not suffer from the percentage of outliers limitation as REPP nP.

In clinical cardiology, both anatomy and physiology are needed to diagnose cardiac pathologies. CT imaging and computer simulations provide valuable and complementary data for this purpose. However, it remains challenging to gain useful information from the large amount of high-dimensional diverse data. The current tools are not adequately integrated to visualize anatomic and physiologic data from a complete yet focused perspective. We introduce a new computer-aided diagnosis framework, which allows for comprehensive modeling and visualization of cardiac anatomy and physiology from CT imaging data and computer simulations, with a primary focus on ischemic heart disease. The following visual information is presented: (1) Anatomy from CT imaging: geometric modeling and visualization of cardiac anatomy, including four heart chambers, left and right ventricular outflow tracts, and coronary arteries; (2) Function from CT imaging: motion modeling, strain calculation, and visualization of four heart chambers; (3) Physiology from CT imaging: quantification and visualization of myocardial perfusion and contextual integration with coronary artery anatomy; (4) Physiology from computer simulation: computation and visualization of hemodynamics (e.g., coronary blood velocity, pressure, shear stress, and fluid forces on the vessel wall). Substantially, feedback from cardiologists have confirmed the practical utility of integrating these features for the purpose of computer-aided diagnosis of ischemic heart disease.

Monocular simultaneous localization and mapping (mono-SLAM) is a key component of autonomous robot visual navigation. Recently, the structure from motion (SfM) approach has become an attractive means of implementing mono-SLAM because of its high accuracy; although, in this application, the SfM approach must be operable in real time and robust to outliers. However, because of strong nonlinearity, conventional SfM methods, such as bundle adjustment, must consider multiple initial values to obtain the globally optimal result, which is time consuming. In this paper, a novel iterative SfM algorithm based on the object-space objective function is proposed. To improve the method's robustness to outliers and to incorporate the information obtained from other types of sensors, an approach to closely integrate the proposed SfM algorithm with extended Kalman filter-based SLAM is proposed. Experimental results using both synthesized and real data are consistent with our theory, and verify that the main advantage of the proposed SfM algorithm is its good convergence. The algorithm is therefore particularly appropriate for realizing mono-SLAM, because when rotations can be obtained approximately using gyroscope information, the algorithm is globally convergent from any initial value.

Pose estimation from points with unknown correspondences currently is still a difficult problem in the field of computer vision. To solve this problem, the SoftSI algorithm is proposed, which can simultaneously obtain pose and correspondences. The SoftSI algorithm is based on the combination of the proposed PnP algorithm (the SI algorithm) and two singular value decomposition (SVD)-based shape description theorems. Other main contributions of this paper are: 1) two SVD-based shape description theorems are proposed; 2) by analyzing the calculation process of the SI algorithm, the method to avoid pose ambiguity is proposed; and 3) an acceleration method to quickly eliminate bad initial values for the SoftSI algorithm is proposed. The simulation results show that the SI algorithm is accurate while the SoftSI algorithm is fast, robust to noise, and has large convergence radius.