In this paper, we propose a robotic ultrasound imaging method that scans the breast in two separate phases to acquire high-quality ultrasound images. Our proposed system controls five Degrees of Freedom (DoFs) of the robot that hold an ultrasound probe to perform precise scanning. This system finds the desired trajectory based on geometrical analysis of the target inside the breast in a pre-scan phase and uses this information to control the probe in a post-scan phase. The proposed method updates the desired values of rotational and translational movement of the probe in the post-scan by calculating the center of mass of segmented target in each acquired frame and the average of image confidence map. The proposed method has been tested experimentally on a plastisol phantom. Given a specific trajectory, the position and orientation of the probe have been controlled at each point of the trajectory. The experiments’ result shows us that our proposed visual servoing algorithm successfully controls the probe to look at target tissue and is fast enough for use in a robotic control loop.

In this paper, a new remote center of motion (RCM) mechanism is presented whose end-effector is able to move through an entire hemisphere. In general, minimally invasive surgery (MIS) applications, an elliptic cone workspace with vertex angles of 60° and 90° gives the surgeon enough freedom to operate. Therefore, the majority of the developed RCM mechanisms have such a cone as the workspace. However, there are still situations in which a larger workspace is required, like the breast ultrasound scanning application in which the RCM mechanisms should be able to move over a hemisphere to do the breast scanning. The proposed RCM mechanism is developed based upon a spherical scissor linkage and benefits from the high stiffness characteristics of parallel structures while eliminating the common problem of linkage collision in parallel structures. It has two rotational degrees of freedom that are decoupled from each other. The Jacobian and the stiffness of the mechanism while considering the bending of the links is calculated through the virtual joints method (VJM). The kinemato-static equations and the methodology for calculating stiffness are described in detail. The optimal arc angle of the mechanism's links is found using a multi-objective genetic algorithm optimization. A prototype of the mechanism is built and forward kinematic of the proposed mechanism is examined experimentally. The experiments indicate that the proposed mechanism is able to provide a hemisphere as its workspace while the RCM point of the mechanism is fixed in the space.

Robotic-assisted needle steering can enhance the accuracy of needle-based interventions. Application of current needle steering techniques are restricted by the limited deflection curvature of needles. Here, a novel steerable needle with improved curvature is developed and used with an online motion planner to steer the needle along curved paths inside tissue. The needle is developed by carving series of small notches on the shaft of a standard needle. The notches decrease the needle flexural stiffness, allowing the needle to follow tightly curved paths with small radius of curvature. In this paper, first, a finite element model of the notched needle deflection in tissue is presented. Next, the model is used to estimate the optimal location for the notches on needle's shaft for achieving a desired curvature. Finally, an ultrasound-guided motion planner for needle steering inside tissue is developed and used to demonstrate the capability of the notched needle in achieving high curvature and maneuvering around obstacles in tissue. We simulated a clinical scenario in brachytherapy, where the target is obstructed by the pubic bone and cannot be reached using regular needles. Experimental results show that the target can be reached using the notched needle with a mean accuracy of 1.2 mm. Thus, the proposed needle enables future research on needle steering toward deeper or more difficult-to-reach targets.

In this paper, we propose an automated method to reconstruct the three-dimensional (3-D) needle shape during needle insertion procedures using only 2-D transverse ultrasound (US) images. Using a set of transverse US images, image processing and random sample consensus are used to locate the needle within each image and estimate the needle shape. The method is validated with an in vitro needle insertion setup and a transparent tissue phantom, where two orthogonal cameras are used to capture the true 3-D needle shape for verification. Results showed that the use of at least three images obtained at 75% of the maximum insertion depth or greater allows for maximum needle shape estimation errors of less than 2 mm. In addition, the needle shape can be calculated consistently as long as the needle can be identified in 30% of the transverse US images obtained. Application to permanent prostate brachytherapy is also presented, where the estimated needle shape is compared to manual segmentation and sagittal US images. Our method is intended to help to assess needle placement during manual or robot-assisted needle insertion procedures after the needle has been inserted.

We propose a method to estimate the entire shape of a long flexible needle, suitable for a needle insertion assistant robot. This method bases its prediction on only a small segment of a needle, imaged via ultrasound, after insertion. An algorithm is developed that can segment a needle observed partially in ultrasound images and fully in camera images, returning a polynomial representation of the needle shape after RANSAC processing. The polynomial corresponding to the partial needle observation in ultrasound images is used as the input to a needle-tissue interaction model that predicts the entire needle shape. The needle shape predicted by the model is compared to the segmented needle shape based on camera images to validate the proposed approach. The results show that the entire needle shape can be accurately predicted in tissues of varying stiffness based on observation of parts of the needle in an ultrasound image.

In needle-based medical procedures, beveled-tip flexible needles are steered inside soft tissue with the aim of reaching pre-defined target locations. The efficiency of needle-based interventions depends on accurate control of the needle tip. This paper presents a comprehensive mechanics-based model for simulation of planar needle insertion in soft tissue. The proposed model for needle deflection is based on beam theory, works in real-time, and accepts the insertion velocity as an input that can later be used as a control command for needle steering. The model takes into account the effects of tissue deformation, needle-tissue friction, tissue cutting force, and needle bevel angle on needle deflection. Using a robot that inserts a flexible needle into a phantom tissue, various experiments are conducted to separately identify different subsets of the model parameters. The validity of the proposed model is verified by comparing the simulation results to the empirical data. The results demonstrate the accuracy of the proposed model in predicting the needle tip deflection for different insertion velocities.

A tissue-independent model to estimate needle deflection during insertion in soft tissue is presented in this paper. A force/torque sensor is connected to the needle base in order to measure forces and moments during insertion due to needle deflection. A static mechanical model, which is based on the Euler-Bernoulli beam equation and the balance of forces applied by the tissue onto the needle takes these force and moment measurements as input. The needle tip deflection can then be calculated based on the beam model undergoing these forces. Three different needle-tissue interaction models are presented. Their estimation performance is evaluated and experimentally compared by carrying out insertion experiments into phantom tissue. The experimental results show a precise estimate of needle tip deflection for a novel virtual sensor introduced in this work. The main advantage of this virtual sensor approach is that measurements obtained from the force/torque sensor are the only necessary model inputs. Furthermore, the approach does not rely on ultrasound or other image-based needle observation techniques. This makes the virtual sensor suitable for real-time feedback of needle tip deflection.

This paper introduces an automatic method to visualize 3D needle shapes for reliable assessment of needle placement during needle insertion procedures. Based on partial observations of the needle within a small sample of 2D transverse ultrasound images, the 3D shape of the entire needle is reconstructed. An intensity thresholding technique is used to identify points representing possible needle locations within each 2D ultrasound image. Then, a Random Sample and Consensus (RANSAC) algorithm is used to filter out false positives and fit the remaining points to a polynomial model. To test this method, a set of 21 transverse ultrasound images of a brachytherapy needle embedded within a transparent tissue phantom are obtained and used to reconstruct the needle shape. Results are validated using camera images which capture the true needle shape. For this experimental data, obtaining at least three images from an insertion depth of 50 mm or greater allows the entire needle shape to be calculated with an average error of 0.5 mm with respect to the measured needle curve obtained from the camera image. Future work and application to robotics is also discussed.

This paper represents an extension to the kinematic bicycle model for beveled-tip needle motion in soft tissue, which accounts for non-constant curvature paths for the needle tip. For a tissue that is not stiff relative to the needle, the tissue deformation caused by needle insertion deviates the needle tip position from a constant curvature path. The proposed model is obtained by replacing the bicycle wheels with omnidirectional wheels that move in two orthogonal directions independently. Such wheels can move sideways, providing a means for modeling the deviations of the needle tip from a constant curvature path by incorporating new parameters in the model. Using an experimental setup, the needle is inserted into soft phantom tissue at different constant velocities and model parameters are fitted to experimental data. The model is verified by comparing the results from the model to empirical data.

This paper presents a comparison of three-dimensional (3-D) segmentations of the prostate, based on two-dimensional (2-D) manually segmented contours, obtained using ultrasound (US) and magnetic resonance (MR) imaging data collected from 40 patients diagnosed with localized prostate cancer and scheduled to receive brachytherapy treatment. The approach we propose here for 3-D prostate segmentation first uses radial basis function interpolation to construct a 3-D point distribution model for each prostate. Next, a modified principal axis transformation is utilized for rigid registration of the US and MR images of the same prostate in preparation for the following shape comparison. Then, statistical shape models are used to capture the segmented 3-D prostate geometries for the subsequent cross-modality comparison. Our study includes not only cross-modality geometric comparisons in terms of prostate volumes and dimensions, but also an investigation of interchangeability of the two imaging modalities in terms of automatic contour segmentation at the pre-implant planning stage of prostate brachytherapy treatment. By developing a new scheme to compare the two imaging modalities in terms of the segmented 3-D shapes, we have taken a first step necessary for building coupled US-MR segmentation strategies for prostate brachytherapy pre-implant planning, which at present is predominantly informed by US images only.