This article describes a novel system for quantitative and volumetric measurement of tissue elasticity in the prostate using simultaneous multi-frequency tissue excitation. Elasticity is computed by using a local frequency estimator to measure the three-dimensional local wavelengths of steady-state shear waves within the prostate gland. The shear wave is created using a mechanical voice coil shaker which transmits simultaneous multi-frequency vibrations transperineally. Radio frequency data is streamed directly from a BK Medical 8848 transrectal ultrasound transducer to an external computer where tissue displacement due to the excitation is measured using a speckle tracking algorithm. Bandpass sampling is used that eliminates the need for an ultra-fast frame rate to track the tissue motion and allows for accurate reconstruction at a sampling frequency that is below the Nyquist rate. A roll motor with computer control is used to rotate the transducer and obtain 3D data. Two commercially available phantoms were used to validate both the accuracy of the elasticity measurements as well as the functional feasibility of using the system for in vivo prostate imaging. The phantom measurements were compared with 3D Magnetic Resonance Elastography (MRE), where a high correlation of 96% was achieved. In addition, the system has been used in two separate clinical studies as a method for cancer identification. Qualitative and quantitative results of 11 patients from these clinical studies are presented here. Furthermore, an AUC of 0.87±0.12 was achieved for malignant vs. benign classification using a binary support vector machine classifier trained with data from the latest clinical study with leave one patient out cross-validation.

The attention mechanism in artificial neural networks is conceptually interlinked with human visual attention, and studies have shown that either artificial or human attention can facilitate computer vision tasks. However, little research has investigated the visual interpretation of neural network’s attention with the help of human visual attention.

In shear wave absolute vibro-elastography (S-WAVE), a steady-state multi-frequency external mechanical excitation is applied to tissue, while a time-series of ultrasound radio-frequency (RF) data are acquired. Our objective is to determine the potential of S-WAVE to classify breast tissue lesions as malignant or benign. We present a new processing pipeline for feature-based classification of breast cancer using S-WAVE data, and we evaluate it on a new data set collected from 40 patients. Novel bi-spectral and Wigner spectrum features are computed directly from the RF time series and are combined with textural and spectral features from B-mode and elasticity images. The Random Forest permutation importance ranking and the Quadratic Mutual Information methods are used to reduce the number of features from 377 to 20. Support Vector Machines and Random Forest classifiers are used with leave-one-patient-out and Monte Carlo cross-validations. Classification results obtained for different feature sets are presented. Our best results (95% confidence interval, Area Under Curve = 95%±1.45%, sensitivity = 95%, and specificity = 93%) outperform the state-of-the-art reported S-WAVE breast cancer classification performance. The effect of feature selection and the sensitivity of the above classification results to changes in breast lesion contours is also studied. We demonstrate that time-series analysis of externally vibrated tissue as an elastography technique, even if the elasticity is not explicitly computed, has promise and should be pursued with larger patient datasets. Our study proposes novel directions in the field of elasticity imaging for tissue classification.

This paper presents U-LanD, a framework for automatic detection of landmarks on key frames of the video by leveraging the uncertainty of landmark prediction. We tackle a specifically challenging problem, where training labels are noisy and highly sparse. U-LanD builds upon a pivotal observation: a deep Bayesian landmark detector solely trained on key video frames, has significantly lower predictive uncertainty on those frames vs. other frames in videos. We use this observation as an unsupervised signal to automatically recognize key frames on which we detect landmarks. As a test-bed for our framework, we use ultrasound imaging videos of the heart, where sparse and noisy clinical labels are only available for a single frame in each video. Using data from 4,493 patients, we demonstrate that U-LanD can exceedingly outperform the state-of-the-art non-Bayesian counterpart by a noticeable absolute margin of 42% in${R}^{{2}}$score, with almost no overhead imposed on the model size.

We introduce two model-based iterative methods to obtain shear modulus images of tissue using magnetic resonance elastography. The first method jointly finds the displacement field that best fits tissue displacement data and the corresponding shear modulus. The displacement satisfies a viscoelastic wave equation constraint, discretized using the finite element method. Sparsifying regularization terms in both shear modulus and displacement are used in the cost function minimized for the best fit. The second method extends the first method for multifrequency tissue displacement data. The formulated problems are bi-convex. Their solution can be obtained iteratively by using the alternating direction method of multipliers. Sparsifying regularizations and the wave equation constraint filter out sensor noise and compressional waves. Our methods do not require bandpass filtering as a preprocessing step and converge fast irrespective of the initialization. We evaluate our new methods in multiple in silico and phantom experiments, with comparisons with existing methods, and we show improvements in contrast to noise and signal-to-noise ratios. Results from an in vivo liver imaging study show elastograms with mean elasticity comparable to other values reported in the literature.

Quantitative ultrasound (QUS) offers a non-invasive and objective way to quantify tissue health. We recently presented a spatially adaptive regularization method for reconstruction of a single QUS parameter, limited to a two dimensional region. That proof-of-concept study showed that regularization using homogeneity prior improves the fundamental precision-resolution trade-off in QUS estimation. Based on the weighted regularization scheme, we now present a multiparametric 3D weighted QUS (3D QUS) method, involving the reconstruction of three QUS parameters: attenuation coefficient estimate (ACE), integrated backscatter coefficient (IBC) and effective scatterer diameter (ESD). With the phantom studies, we demonstrate that our proposed method accurately reconstructs QUS parameters, resulting in high reconstruction contrast and therefore improved diagnostic utility. Additionally, the proposed method offers the ability to analyze the spatial distribution of QUS parameters in 3D, which allows for superior tissue characterization. We apply a three-dimensional total variation regularization method for the volumetric QUS reconstruction. The 3D regularization involving N planes results in a high QUS estimation precision, with an improvement of standard deviation over the theoretical 1/N rate achievable by compounding N independent realizations. In the in vivo liver study, we demonstrate the advantage of adopting a multiparametric approach over the single parametric counterpart, where a simple quadratic discriminant classifier using feature combination of three QUS parameters was able to attain a perfect classification performance to distinguish between normal and fatty liver cases.

In echocardiography (echo), an electrocardiogram (ECG) is conventionally used to temporally align different cardiac views for assessing critical measurements. However, in emergencies or point-of-care situations, acquiring an ECG is often not an option, hence motivating the need for alternative temporal synchronization methods. Here, we propose Echo-SyncNet, a self-supervised learning framework to synchronize various cross-sectional 2D echo series without any human supervision or external inputs. The proposed framework takes advantage of two types of supervisory signals derived from the input data: spatiotemporal patterns found between the frames of a single cine (intra-view self-supervision) and interdependencies between multiple cines (inter-view self-supervision). The combined supervisory signals are used to learn a feature-rich and low dimensional embedding space where multiple echo cines can be temporally synchronized. Two intra-view self-supervisions are used, the first is based on the information encoded by the temporal ordering of a cine (temporal intra-view) and the second on the spatial similarities between nearby frames (spatial intra-view). The inter-view self-supervision is used to promote the learning of similar embeddings for frames captured from the same cardiac phase in different echo views. We evaluate the framework with multiple experiments: 1) Using data from 998 patients, Echo-SyncNet shows promising results for synchronizing Apical 2 chamber and Apical 4 chamber cardiac views, which are acquired spatially perpendicular to each other; 2) Using data from 3070 patients, our experiments reveal that the learned representations of Echo-SyncNet outperform a supervised deep learning method that is optimized for automatic detection of fine-grained cardiac cycle phase; 3) We go one step further and show the usefulness of the learned representations in a one-shot learning scenario of cardiac key-frame detection. Without any fine-tuning, key frames in 1188 validation patient studies are identified by synchronizing them with only one labeled reference cine. We do not make any prior assumption about what specific cardiac views are used for training, and hence we show that Echo-SyncNet can accurately generalize to views not present in its training set. Project repository: github.com/fatemehtd/Echo-SyncNet>.

Magnetic resonance elastography (MRE) is commonly regarded as the imaging-based gold-standard for liver fibrosis staging, comparable to biopsy. While ultrasound-based elastography methods for liver fibrosis staging have been developed, they are confined to a 1D or a 2D region of interest and to a limited depth. 3D Shear Wave Absolute Vibro-Elastography (S-WAVE) is a steady-state, external excitation, volumetric elastography technique that is similar to MRE, but has the additional advantage of multi-frequency excitation. We present a novel ultrasound matrix array implementation of S-WAVE that takes advantage of 3D imaging. We use a matrix array transducer to sample axial multi-frequency steady-state tissue motion over a volume, using a Color Power Angiography sequence. Tissue motion with the frequency components (40, 50,60) and (45, 55, 65) Hz are acquired over a (90° lateral)×(40° elevational)×(16 cm depth) sector with an acquisition time of 12 seconds. We compute the elasticity map in 3D using local spatial frequency estimation. We characterize this new approach in tissue phantoms against measurements obtained with transient-elastography and MRE. Six healthy volunteers and eight patients with chronic liver disease were imaged. Their MRE and S-WAVE volumes were aligned using T1 to B-mode registration for direct comparison in common regions of interest. S-WAVE and MRE results are correlated with R2= 0.92, while MRE and TE results are correlated with R2= 0.71. Our findings show that S-WAVE with matrix array has the potential to deliver a similar assessment of liver fibrosis as MRE in a more accessible, inexpensive way, to a broader set of patients.

Uncertainty of labels in clinical data resulting from intra-observer variability can have direct impact on the reliability of assessments made by deep neural networks. In this paper, we propose a method for modelling such uncertainty in the context of 2D echocardiography (echo), which is a routine procedure for detecting cardiovascular disease at point-of-care. Echo imaging quality and acquisition time is highly dependent on the operator's experience level. Recent developments have shown the possibility of automating echo image quality quantification by mapping an expert's assessment of quality to the echo image via deep learning techniques. Nevertheless, the observer variability in the expert's assessment can impact the quality quantification accuracy. Here, we aim to model the intra-observer variability in echo quality assessment as an aleatoric uncertainty modelling regression problem with the introduction of a novel method that handles the regression problem with categorical labels. A key feature of our design is that only a single forward pass is sufficient to estimate the level of uncertainty for the network output. Compared to the 0.11 ± 0.09 absolute error (in a scale from 0 to 1) archived by the conventional regression method, the proposed method brings the error down to 0.09 ± 0.08, where the improvement is statistically significant and equivalents to 5.7% test accuracy improvement. The simplicity of the proposed approach means that it could be generalized to other applications of deep learning in medical imaging, where there is often uncertainty in clinical labels.

Accurate detection of end-systolic (ES) and end-diastolic (ED) frames in an echocardiographic cine series can be difficult but necessary pre-processing step for the development of automatic systems to measure cardiac parameters. The detection task is challenging due to variations in cardiac anatomy and heart rate often associated with pathological conditions. We formulate this problem as a regression problem and propose several deep learning-based architectures that minimize a novel global extrema structured loss function to localize the ED and ES frames. The proposed architectures integrate convolution neural networks (CNNs)-based image feature extraction model and recurrent neural networks (RNNs) to model temporal dependencies between each frame in a sequence. We explore two CNN architectures: DenseNet and ResNet, and four RNN architectures: long short-term memory, bi-directional LSTM, gated recurrent unit (GRU), and Bi-GRU, and compare the performance of these models. The optimal deep learning model consists of a DenseNet and GRU trained with the proposed loss function. On average, we achieved 0.20 and 1.43 frame mismatch for the ED and ES frames, respectively, which are within reported inter-observer variability for the manual detection of these frames.

Current deep supervised learning methods typically require large amounts of labeled data for training. Since there is a significant cost associated with clinical data acquisition and labeling, medical datasets used for training these models are relatively small in size. In this paper, we aim to alleviate this limitation by proposing a variational generative model along with an effective data augmentation approach that utilizes the generative model to synthesize data. In our approach, the model learns the probability distribution of image data conditioned on a latent variable and the corresponding labels. The trained model can then be used to synthesize new images for data augmentation. We demonstrate the effectiveness of the approach on two independent clinical datasets consisting of ultrasound images of the spine and magnetic resonance images of the brain. For the spine dataset, a baseline and a residual model achieve an accuracy of 85% and 92%, respectively, using our method compared to 78% and 83% using a conventional training approach for image classification task. For the brain dataset, a baseline and a U-net network achieve an accuracy of 84% and 88%, respectively, in Dice coefficient in tumor segmentation compared to 80% and 83% for the convention training approach.

Accurate identification of the needle target is crucial for effective epidural anesthesia. Currently, epidural needle placement is administered by a manual technique, relying on the sense of feel, which has a significant failure rate. Moreover, misleading the needle may lead to inadequate anesthesia, post dural puncture headaches, and other potential complications. Ultrasound offers guidance to the physician for identification of the needle target, but accurate interpretation and localization remain challenges. A hybrid machine learning system is proposed to automatically localize the needle target for epidural needle placement in ultrasound images of the spine. In particular, a deep network architecture along with a feature augmentation technique is proposed for automatic identification of the anatomical landmarks of the epidural space in ultrasound images. Experimental results of the target localization on planes of 3-D as well as 2-D images have been compared against an expert sonographer. When compared with the expert annotations, the average lateral and vertical errors on the planes of 3-D test data were 1 and 0.4 mm, respectively. On 2-D test data set, an average lateral error of 1.7 mm and vertical error of 0.8 mm were acquired.

We present a novel technique for real-time deformable registration of 3-D to 2.5-D transrectal ultrasound (TRUS) images for image-guided, robot-assisted laparoscopic radical prostatectomy (RALRP). For RALRP, a pre-operatively acquired 3-D TRUS image is registered to thin-volumes comprised of consecutive intra-operative 2-D TRUS images, where the optimal transformation is found using a gradient descent method based on analytical first and second order derivatives. Our method relies on an efficient algorithm for real-time extraction of arbitrary slices from a 3-D image deformed given a discrete mesh representation. We also propose and demonstrate an evaluation method that generates simulated models and images for RALRP by modeling tissue deformation through patient-specific finite-element models (FEM). We evaluated our method on in-vivo data from 11 patients collected during RALRP and focal therapy interventions. In the presence of an average landmark deformation of 3.89 and 4.62 mm, we achieved accuracies of 1.15 and 0.72 mm, respectively, on the synthetic and in-vivo data sets, with an average registration computation time of 264 ms, using MATLAB on a conventional PC. The results show that the real-time tracking of the prostate motion and deformation is feasible, enabling a real-time augmented reality-based guidance system for RALRP.].

Echocardiography (echo) is a skilled technical procedure that depends on the experience of the operator. The aim of this paper is to reduce user variability in data acquisition by automatically computing a score of echo quality for operator feedback. To do this, a deep convolutional neural network model, trained on a large set of samples, was developed for scoring apical four-chamber (A4C) echo. In this paper, 6,916 end-systolic echo images were manually studied by an expert cardiologist and were assigned a score between 0 (not acceptable) and 5 (excellent). The images were divided into two independent training-validation and test sets. The network architecture and its parameters were based on the stochastic approach of the particle swarm optimization on the training-validation data. The mean absolute error between the scores from the ultimately trained model and the expert's manual scores was 0.71 ± 0.58. The reported error was comparable to the measured intra-rater reliability. The learned features of the network were visually interpretable and could be mapped to the anatomy of the heart in the A4C echo, giving confidence in the training result. The computation time for the proposed network architecture, running on a graphics processing unit, was less than 10 ms per frame, sufficient for real-time deployment. The proposed approach has the potential to facilitate the widespread use of echo at the point-of-care and enable early and timely diagnosis and treatment. Finally, the approach did not use any specific assumptions about the A4C echo, so it could be generalizable to other standard echo views.

In the above-title paper [ibid., vol. 36, no. 6, pp. 1221-1230, Jun. 2017], the first footnote should have indicated the following information: A. H. Abdi and C. Luong are joint first authors.

We propose a joint information approach for automatic analysis of 2D echocardiography (echo) data. The approach combines a priori images, their segmentations and patient diagnostic information within a unified framework to determine various clinical parameters, such as cardiac chamber volumes, and cardiac disease labels. The main idea behind the approach is to employ joint Independent Component Analysis of both echo image intensity information and corresponding segmentation labels to generate models that jointly describe the image and label space of echo patients on multiple apical views, instead of independently. These models are then both used for segmentation and volume estimation of cardiac chambers such as the left atrium and for detecting pathological abnormalities such as mitral regurgitation. We validate the approach on a large cohort of echoes obtained from 6,993 studies. We report performance of the proposed approach in estimation of the left-atrium volume and detection of mitral-regurgitation severity. A correlation coefficient of 0.87 was achieved for volume estimation of the left atrium when compared to the clinical report. Moreover, we classified patients that suffer from moderate or severe mitral regurgitation with an average accuracy of 82%.

In quantitative elastography, maps of the mechanical properties of soft tissue, or elastograms, are calculated from the measured displacement data by solving an inverse problem. The model assumptions have a significant effect on elastograms. Motivated by the high sensitivity of imaging results to the model assumptions for in vivo magnetic resonance elastography of the prostate, we compared elastograms obtained with four different methods. Two finite-element method (FEM)-based methods developed by our group were compared with two other commonly used methods, local frequency estimator (LFE) and curl-based direct inversion (c-DI). All the methods assume a linear isotropic elastic model, but the methods vary in their assumptions, such as local homogeneity or incompressibility, and in the specific approach used. We report results using simulations, phantom, and ex vivo and in vivo data. The simulation and phantom studies show, for regions with an inclusion, that the contrast to noise ratio (CNR) for the FEM methods is about three to five times higher than the CNR for the LFE and c-DI and the rms error is about half. The LFE method produces very smooth results (i.e., low CNR) and is fast. c-DI is faster than the FEM methods but it is only accurate in areas where elasticity variations are small. The artifacts resulting from the homogeneity assumption in c-DI is detrimental in regions with large variations. The ex vivo and in vivo results also show similar trends as the simulation and phantom studies. The c-FEM method is more sensitive to noise compared with the mixed-FEM due to higher orders derivatives. This is especially evident at lower frequencies, where the wave curvature is smaller and it is more prone to such error, causing a discrepancy in the absolute values between the mixed-FEM and c-FEM in our in vivo results. In general, the proposed FEMs use fewer simplifying assumptions and outperform the other methods but they are computationally more expensive.

Segmentation of the wrist bones in CT images has been frequently used in different clinical applications including arthritis evaluation, bone age assessment and image-guided interventions. The major challenges include non-uniformity and spongy textures of the bone tissue as well as narrow inter-bone spaces. In this work, we propose an automatic wrist bone segmentation technique for CT images based on a statistical model that captures the shape and pose variations of the wrist joint across 60 example wrists at nine different wrist positions. To establish the correspondences across the training shapes at neutral positions, the wrist bone surfaces are jointly aligned using a group-wise registration framework based on a Gaussian Mixture Model. Principal component analysis is then used to determine the major modes of shape variations. The variations in poses not only across the population but also across different wrist positions are incorporated in two pose models. An intra-subject pose model is developed by utilizing the similarity transforms at all wrist positions across the population. Further, an inter-subject pose model is used to model the pose variations across different wrist positions. For segmentation of the wrist bones in CT images, the developed model is registered to the edge point cloud extracted from the CT volume through an expectation maximization based probabilistic approach. Residual registration errors are corrected by application of a non-rigid registration technique. We validate the proposed segmentation method by registering the wrist model to a total of 66 unseen CT volumes of average voxel size of 0.38 mm. We report a mean surface distance error of 0.33 mm and a mean Jaccard index of 0.86.

This work reports the use of ultrasound radio frequency (RF) time series analysis as a method for ultrasound-based classification of malignant breast lesions. The RF time series method is versatile and requires only a few seconds of raw ultrasound data with no need for additional instrumentation. Using the RF time series features, and a machine learning framework, we have generated malignancy maps, from the estimated cancer likelihood, for decision support in biopsy recommendation. These maps depict the likelihood of malignancy for regions of size 1 mm(2) within the suspicious lesions. We report an area under receiver operating characteristics curve of 0.86 (95% confidence interval [CI]: 0.84%-0.90%) using support vector machines and 0.81 (95% CI: 0.78-0.85) using Random Forests classification algorithms, on 22 subjects with leave-one-subject-out cross-validation. Changing the classification method yielded consistent results which indicates the robustness of this tissue typing method. The findings of this report suggest that ultrasound RF time series, along with the developed machine learning framework, can help in differentiating malignant from benign breast lesions, subsequently reducing the number of unnecessary biopsies after mammography screening.

A big challenge in sensorless image-based ultrasound tracking is in the out-of-plane motion estimation. The correlation value of a specific model of speckle known as fully developed speckle (FDS) can be used to estimate the out-of-plane displacement. In real tissue, this kind of pattern is rare and the deviation of speckle pattern from the ideal FDS model diminishes the accuracy of the out-of-plane motion estimation. In this paper a new method for estimation of the out-of-plane motion is proposed. Firstly a closed-form mathematical derivation is provided for the correlation of two RF echo signal patches at different positions. A linear regression model of the ultrasound beam profile is proposed to account for the spatial variability of the ultrasound beam and enhance the accuracy of out-of-plane motion estimation in real tissue. The statistical model of speckle used here is based on the Rician-Inverse Gaussian (RiIG) stochastic process of the speckle formation, which can be considered as a generalized form of the K-distribution with richer parametrization. In this work, for the first time the second-order statistics of the RIG model is used for speckle tracking. This statistical model allows for derivation of a closed-form formulation for the correlation coefficient based on the statistical parameters of every patch. Since the effect of coherency is considered in the RiIG model, it increases the reliability of the out-of-plane motion estimation. The flexibility of the proposed method enables almost any patch through the whole image to be used for the purpose of displacement estimation. The method has been evaluated both on ex vivo and in vivo tissues in various experiments including out-of-plane rotation (tilt, yaw) and free-hand imaging. The overall outcome demonstrates the potential of the proposed method for in vivo tissues.

Most conventional spine interventions are performed under X-ray fluoroscopy guidance. In recent years, there has been a growing interest to develop nonionizing imaging alternatives to guide these procedures. Ultrasound guidance has emerged as a leading alternative. However, a challenging problem is automatic identification of the spinal anatomy in ultrasound data. In this paper, we propose a local phase-based bone feature enhancement technique that can robustly identify the spine surface in ultrasound images. The local phase information is obtained using a gradient energy tensor filter. This information is used to construct local phase tensors in ultrasound images, which highlight the spine surface. We show that our proposed approach results in a more distinct enhancement of the bone surfaces compared to recently proposed techniques based on monogenic scale-space filters and logarithmic Gabor filters. We also demonstrate that registration accuracy of a statistical shape+pose model of the spine to 3-D ultrasound images can be significantly improved, using the proposed method, compared to those obtained using monogenic scale-space filters and logarithmic Gabor filters.

In elasticity imaging, the shear modulus is obtained from measured tissue displacement data by solving an inverse problem based on the wave equation describing the tissue motion. In most inversion approaches, the wave equation is simplified using local homogeneity and incompressibility assumptions. This causes a loss of accuracy and therefore imaging artifacts in the resulting elasticity images. In this paper we present a new curl-based finite element method inversion technique that does not rely upon these simplifying assumptions. As done in previous research, we use the curl operator to eliminate the dilatational term in the wave equation, but we do not make the assumption of local homogeneity. We evaluate our approach using simulation data from a virtual tissue phantom assuming time harmonic motion and linear, isotropic, elastic behavior of the tissue. We show that our reconstruction results are superior to those obtained using previous curl-based methods with homogeneity assumption. We also show that with our approach, in the 2-D case, multi-frequency measurements provide better results than single-frequency measurements. Experimental results from magnetic resonance elastography of a CIRS elastography phantom confirm our simulation results and further demonstrate, in a quantitative and repeatable manner, that our method is accurate and robust.

Segmentation of the spinal column from computed tomography (CT) images is a preprocessing step for a range of image-guided interventions. One intervention that would benefit from accurate segmentation is spinal needle injection. Previous spinal segmentation techniques have primarily focused on identification and separate segmentation of each vertebra. Recently, statistical multi-object shape models have been introduced to extract common statistical characteristics between several anatomies. These models can be used for segmentation purposes because they are robust, accurate, and computationally tractable. In this paper, we develop a statistical multi-vertebrae shape+pose model and propose a novel registration-based technique to segment the CT images of spine. The multi-vertebrae statistical model captures the variations in shape and pose simultaneously, which reduces the number of registration parameters. We validate our technique in terms of accuracy and robustness of multi-vertebrae segmentation of CT images acquired from lumbar vertebrae of 32 subjects. The mean error of the proposed technique is below 2 mm, which is sufficient for many spinal needle injection procedures, such as facet joint injections.

In this study we evaluated a new method for registering three-dimensional ultrasound (3DUS) data to external coordinate systems. First, 3DUS was registered to the stereo endoscope of a da Vinci Surgical System by placing a registration tool against an air-tissue boundary so that the 3DUS could image ultrasound fiducials while the stereo endoscope could image camera markers on the same tool. The common points were used to solve the registration between the 3DUS and camera coordinate systems. The target registration error (TRE) when imaging through a PVC tissue phantom ranged from 3.85 1.76 mm to 1.82 1.03 mm using one to four registration tool positions. TRE when imaging through an ex-vivo liver tissue sample ranged from 2.36 1.01 mm to 1.51 0.70 mm using one to four registration tool positions. Second, using a similar method, 3DUS was registered to the kinematic coordinate system of a da Vinci Surgical System by using the da Vinci surgical manipulators to identify common points on an air-tissue boundary. TRE when imaging through a PVC tissue phantom was 0.95 0.38 mm. This registration method is simpler and potentially more accurate than methods using commercial motion tracking systems. This method may be useful in the future in augmented reality systems for laparoscopic and robotic-assisted surgery.

This paper presents a novel, fast, group-wise registration technique based on establishing soft correspondences between groups of point sets. The registration approach is used to create a statistical shape model, capable of learning the shape variations within a training set. The shape model consists of a mean shape and its transformations to all training shapes. We decouple the procedure into two steps: updating the mean shape and registering it to the training shapes. The algorithm alternates between these two steps until convergence. Following the generation of the statistical shape model, we use the soft correspondence approach to register the model to a new observation. We perform extensive experiments on two data sets: lumbar spine and hippocampi. We compare our algorithm to available state-of- the-art group-wise registration algorithms including feature-based and volume-based approaches. We demonstrate improved generalization, specificity and compactness compared to these algorithms.

Vision-based tracking of tissue is a key component to enable augmented reality during a surgical operation. Conven- tional tracking techniques in computer vision rely on identifying strong edge features or distinctive textures in a well-lit environ- ment; however endoscopic tissue images do not have strong edge features, are poorly lit and exhibit a high degree of specular reflection. Therefore, prior work in achieving densely populated 3D features for describing tissue surface profiles require complex image processing techniques and have been limited in providing stable, long-term tracking or real-time processing. In this paper, we present an integrated framework for ac- curately tracking tissue in surgical stereo-cameras at real-time speeds. We use a combination of the STAR feature detector and Binary Robust Independent Elementary Features to acquire salient features that can be persistently tracked at high frame rates. The features are then used to acquire a densely-populated map of the deformations of tissue surface in 3D. We evaluate the method against popular feature algorithms in in-vivo animal study video sequences, and we also apply the proposed method to human partial nephrectomy video sequences. We extend the salient feature framework to support region tracking in order to maintain the spatial correspondence of a tracked region of tissue or a medical image registration to the surrounding tissue. In-vitro tissue studies show registration accuracies of 1.3-3.3 mm using a rigid-body transformation method.

In this paper, a novel approach to the problem of elasticity reconstruction is introduced. In this approach, the solution of the wave equation is expanded as a sum of waves travelling in different directions sharing a common wave number. In particular, the solutions for the scalar and vector potentials which are related to the dilatational and shear components of the displacement respectively are expanded as sums of travelling waves. This solution is then used as a model and fitted to the measured displacements. The value of the shear wave number which yields the best fit is then used to find the elasticity at each spatial point. The main advantage of this method over direct inversion methods is that, instead of taking the derivatives of noisy measurement data, the derivatives are taken on the analytical model. This improves the results of the inversion. The dilatational and shear components of the displacement can also be computed as a byproduct of the method, without taking any derivatives. Experimental results show the effectiveness of this technique in magnetic resonance elastography. Comparisons are made with other state-of-the-art techniques.

The goal of this research is to use eyegaze tracking data to provide insights into designing radiology workstations. We designed a look-alike radiology task with artificial stimuli. The task involved a comparative visual search of two side-by-side images, using two different interaction techniques. We tracked the eyegaze of four radiologists while they performed the task and measured the duration of the fixations on the controls, the left and right images, and on the artificial targets. Response time differences between the two interaction techniques exceeded the differences of fixations on the controls. Fixations on the left-side images are longer than the right-side images, and the search for multifeatured targets occurs in two phases: first a regular scan path search phase for a likely target and then a confirmation phase of several fixations on the target in each side-by-side image. We conclude that eyegaze tracking shows that disruption of visual search leads to cognitive disruption; subjects use the left image as a reference image and multiple saccades between left and right side images are necessary, because of the limitations of the visual working memory.

This paper presents a new technique for the extraction of surfaces from 3D ultrasound data. Surface extraction from ultrasound data is challenging for a number of reasons including noise and artifacts in the images and nonuniform data sampling. A method is proposed to fit an approximating radial basis function to the group of data samples. An explicit surface is then obtained by iso-surfacing the function. In most previous 3D ultrasound research, a pre-processing step is taken to interpolate the data into a regular voxel array and a corresponding loss of resolution. We are the first to represent the set of semi-structured ultrasound pixel data as a single function. From this we are able to extract surfaces without first reconstructing the irregularly spaced pixels into a regular 3D voxel array.

One of the most promising applications of 3-D ultrasound lies in the visualisation and volume estimation of internal 3-D structures. Unfortunately, artifacts and speckle make automatic analysis of the data difficult. In this paper we investigate the use of 3-D spatial compounding to improve data quality, and find that accurate registration is the key. A correlation-based registration technique is applied to 3-D ultrasound data acquired from in-vivo examinations of a human gall bladder. We find that the registration technique performs well, and visualisation and segmentation of the compounded data are clearly improved.

3-D ultrasound is a new medical imaging technique which can quickly and non-invasively deliver clinically useful renderings of internal structures. The technique is vulnerable to tissue motion during the scan which can result in an inconsistent 3-D data set. This paper describes an algorithm which corrects for tissue motion and other errors that cause mis-registration during a scan. The resulting 3-D data sets are clearer and of enhanced diagnostic utility. The registration is achieved by applying visual tracking techniques outside their traditional area of application. The algorithm is automatic, incremental and easily incorporated into existing 3-D ultrasound systems.

Precise teleoperation of dextrous robotic hands by hand masters requires an accurate human hand model. A kinematic model of a human index finger is developed as an example for human hand modeling. The parameters of the model are determined by open-loop kinematic calibration. Singular value decomposition is used as a tool for analyzing the kinematic model and the identification process. Accurate and reliable results are obtained only when the numerical condition is minimized through parameter scaling, model reduction and pose set selection. The identified kinematic parameters show the kinematic model and calibration procedure have an accuracy on the order of a few millimetres.>