Registering pre-operative modalities, such as magnetic resonance imaging or computed tomography, to ultrasound images is crucial for guiding clinicians during surgeries and biopsies. Recently, deep-learning approaches have been proposed to increase the speed and accuracy of this registration problem. However, all of these approaches need expensive supervision from the ultrasound domain. In this work, we propose a multitask generative framework that needs weak supervision only from the pre-operative imaging domain during training. To perform a deformable registration, the proposed framework translates a magnetic resonance image to the ultrasound domain while preserving the structural content. To demonstrate the efficacy of the proposed method, we tackle the registration problem of pre-operative 3D MR to transrectal ultrasonography images as necessary for targeted prostate biopsies. We use an in-house dataset of 600 patients, divided into 540 for training, 30 for validation, and the remaining for testing. An expert manually segmented the prostate in both modalities for validation and test sets to assess the performance of our framework. The proposed framework achieves a 3.58 mm target registration error on the expert-selected landmarks, 89.2% in the Dice score, and 1.81 mm 95th percentile Hausdorff distance on the prostate masks in the test set. Our experiments demonstrate that the proposed generative model successfully translates magnetic resonance images into the ultrasound domain. The translated image contains the structural content and fine details due to an ultrasound-specific two-path design of the generative model. The proposed framework enables training learning-based registration methods while only weak supervision from the pre-operative domain is available.

Catheter tracking has become an integral part of interventional radiology. Over the last decades, researchers have significantly contributed to theoretical and technical catheter tracking solutions. However, most of the published work thus far focuses on a single application or a single tracking technology. This paper provides an exhaustive review of the state-of-the-art for catheter tracking in general by analyzing significant contributions in this field. We first present a historical overview that led to catheter tracking and continue with a survey of leading tracking technologies. These include image-based tracking, active and passive tracking, electromagnetic tracking, fiber optic shape sensing, bioelectric navigation, robotic tracking solutions, and hybrid tracking. As for imaging modalities, the focus is on x-ray based modalities, ultrasound, and magnetic resonance imaging. Finally, we review each tracking technology with respect to the imaging modality and establish the relation between the two and the underlying anatomy of interest.

The non-invasive quantification of the cerebral metabolic rate for glucose (CMRGlc) and the characterization of cerebral metabolism in the cerebrovascular territories are helpful in understanding ischemic cerebrovascular disease (ICVD). Firstly, we investigated a non-invasive quantification approach based on an image-derived input function (IDIF) in ICVD. Second, we studied the metabolic changes in CMRGlc after surgical intervention. We evaluated the hypothesis that the IDIF method based on the unilateral internal carotid artery could address challenges in ICVD quantification. The CMRGlc and standardized uptake value ratio (SUVR) were used to measure glucose metabolism activity. Healthy controls showed no significant differences in CMRGlc values between bilateral and unilateral IDIF measurements (intraclass correlation coefficient [ICC]: 0.91-0.98). Patients with ICVD showed significantly increased CMRGlc values after surgical intervention for all territories (percentage changes: 7.4%-22.5%). In contrast, SUVR showed minor differences between postoperative and preoperative patients, indicating that it was a poor biomarker for the diagnosis of ICVD. A significant association between CMRGlc and the National Institutes of Health Stroke Scale (NIHSS) scores was observed (r=-0.54). Our findings suggested that IDIF could be a valuable tool for CMRGlc quantification in patients with ICVD and may advance personalized precision interventions.

Robotic three-dimensional (3D) ultrasound (US) imaging has been employed to overcome the drawbacks of traditional US examinations, such as high inter-operator variability and lack of repeatability. However, object movement remains a challenge as unexpected motion decreases the quality of the 3D compounding. Furthermore, attempted adjustment of objects, e.g., adjusting limbs to display the entire limb artery tree, is not allowed for conventional robotic US systems. To address this challenge, we propose a vision-based robotic US system that can monitor the object’s motion and automatically update the sweep trajectory to provide 3D compounded images of the target anatomy seamlessly. To achieve these functions, a depth camera is employed to extract the manually planned sweep trajectory after which the normal direction of the object is estimated using the extracted 3D trajectory. Subsequently, to monitor the movement and further compensate for this motion to accurately follow the trajectory, the position of firmly attached passive markers is tracked in real-time. Finally, a stepwise compounding was performed. The experiments on a gel phantom demonstrate that the system can resume a sweep when the object is not stationary during scanning.

Within image-guided surgery, `hybrid' guidance technologies have been used to integrate the complementary features of radioactive guidance and fluorescence guidance. Here, we explore how the generation of a novel freehand fluorescence (fhFluo) imaging approach complements freehand SPECT (fhSPECT) in a hybrid setup. Near-infrared optical tracking was used to register the position and the orientation of a hybrid opto-nuclear detection probe while recording its readings. Dedicated look-up table models were used for 3D reconstruction. In phantom and excised tissue settings (i.e., flat-surface human skin explants), fhSPECT and fhFluo were investigated for image resolution and in-tissue signal penetration. Finally, the combined potential of these freehand technologies was evaluated on prostate and lymph node specimens of prostate cancer patients receiving prostatectomy and sentinel lymph node dissection (tracers: indocyanine green (ICG) +99mTc-nanocolloid or ICG-99mTc-nanocolloid). After hardware and software integration, the hybrid setup created 3D nuclear and fluorescence tomography scans. The imaging resolution of fhFluo (1 mm) was superior to that of fhSPECT (6 mm). Fluorescence modalities were confined to a maximum depth of 0.5 cm, while nuclear modalities were usable at all evaluated depths (<; 2 cm). Both fhSPECT and fhFluo enabled augmented- and virtual-reality navigation toward segmented image hotspots, including relative hotspot quantification with an accuracy of 3.9% and 4.1%. Imaging in surgical specimens confirmed these trends (fhSPECT: in-depth detectability, low resolution, and fhFluo: superior resolution, superficial detectability). Overall, when radioactive and fluorescent tracer signatures are used, fhFluo has complementary value to fhSPECT. Combined the freehand technologies render a unique hybrid imaging and navigation modality.

In this paper we present the usage of a drop-in gamma probe for intra-operative Single-Photon Emission Computed Tomography (SPECT) imaging in the scope of minimally invasive robot-assisted interventions. The probe is designed to be inserted and reside inside the abdominal cavity during the intervention. It is grasped during the procedure using a robotic laparoscopic gripper enabling full six degrees of freedom handling by the surgeon. We demonstrate the first deployment of the tracked probe for intra-operative in-patient robotic SPECT enabling augmented-reality image guidance. The hybrid mechanical- and image-based in-patient probe tracking is shown to have an accuracy of 0.2 mm. The overall system performance is evaluated and tested with a phantom for gynecological sentinel lymph node interventions and compared to ground-truth data yielding a mean reconstruction accuracy of 0.67 mm.

For the past two decades, medical Augmented Reality visualization has been researched and prototype systems have been tested in laboratory setups and limited clinical trials. Up to our knowledge, until now, no commercial system incorporating Augmented Reality visualization has been developed and used routinely within the real-life surgical environment. In this paper, we are reporting on observations and analysis concerning the usage of a commercially developed and clinically approved Freehand SPECT system, which incorporates monitor-based Mixed Reality visualization, during real-life surgeries. The workflow-based analysis we present is focused on an atomic sub-task of sentinel lymph node biopsy. We analyzed the usage of the Augmented and Virtual Reality visualization modes by the surgical team, while leaving the staff completely uninfluenced and unbiased in order to capture the natural interaction with the system. We report on our observations in over 100 Freehand SPECT acquisitions within different phases of 52 surgeries.