Flexible endoscope motion tracking and analysis in mechanical simulators have proven useful for endoscopy training. Common motion tracking methods based on electromagnetic tracker are however limited by their high cost and material susceptibility. In this work, the motion-guided dual-camera vision tracker is proposed to provide robust and accurate tracking of the endoscope tip's 3D position. The tracker addresses several unique challenges of tracking flexible endoscope tip inside a dynamic, life-sized mechanical simulator. To address the appearance variation and keep dualcamera tracking consistency, the cross-camera mutual template strategy (CMT) is proposed by introducing dynamic transient mutual templates. To alleviate large occlusion and light-induced distortion, the Mamba-based motion-guided prediction head (MMH) is presented to aggregate historical motion with visual tracking. The proposed tracker achieves superior performance against state-of-the-art vision trackers, achieving 42% and 72% improvements against the second-best method in average error and maximum error. Further motion analysis involving novice and expert endoscopists also shows that the tip 3D motion provided by the proposed tracker enables more reliable motion analysis and more substantial differentiation between different expertise levels, compared with other trackers. Project page: https://github.com/PieceZhang/MotionDCTrack

Ultrasound (US) is widely used in image-guided needle procedures. Correctly tracking the needle tip position in US images during the procedure plays an important role in improving the needle targeting accuracy and patient safety. This paper presents a leaning-based visual tracking network with a Siamese architecture, which makes full use of the attention mechanism to explore the potential of global features and takes advantage of an online target model prediction module to robustly track the needle tip in US images. Several self- and cross-attention modules are applied to learn global features from the whole US image. A discriminative target model is also learned as a complementary part to improve the discriminability of the proposed tracker. The template used during the tracking is updated frequently according to the tracking results to ensure that the tracker can always capture the latest characteristics of the appearance of the needle tip. Experimental results in both phantom and tissue showed that the proposed tracking network was more robust than other state-of-the-art visual trackers. The mean success rates of the proposed tracker are 7.1% and 9.2% higher than the second best performing visual tacker when the needle was inserted by motors and human hands in the tissue experiments.

Magnetic resonance imaging (MRI) is increasingly used for robotic needle-based clinical diagnosis and therapy due to its high resolution and high soft tissue contrast. A needle driver therefore becomes an essential part of these MRI -guided robotic systems to perform in-bore needle placement under continuous MR imaging. However, existing actuator designs for needle drivers are constrained by the high magnetic field and the bore size of the MR scanner, and most lack high- performance capability, especially in terms of large output force and long stroke. In this work, we introduce an MR-safe concentric bellows linear actuator (CBLA) with improved performance over the existing designs. This soft actuator achieves 45N output force, 77 % stroke-length ratio for the bellows, and around 250% stroke-diameter ratio. A mathematical model was built to characterize the behavior of the actuator to provide guidance for the actuator design. The fabrication and experimental evaluation of the actuator are also presented. The results demonstrate the effectiveness of the CBLA design and the strong potential of its application in MR-guided surgical procedures.

Pericardiocentesis is an important surgical intervention to treat a medical condition called pericardial effusion, during which excessive fluid accumulates around the heart, potentially leading to life-threatening situation. It involves the insertion of a needle and catheter towards the heart into the pericardial space to drain the excessive fluid under ultrasound (US) guidance. The risky procedure requires surgeons to acquire sufficient training to ensure safe execution of the procedure. However, existing heart simulators lack dynamic features, do not offer realistic images under US imaging, and are not reusable. This work presents a dynamic heart simulator (DHS) with pericardial effusion to mimic the beating motion of the human heart and the realistic US imaging results. The beating heart motion is realized using a hydraulic actuation system connected to a double-layer balloon set. The clear and realistic US imaging results are obtained through a unique formula proposed for the chest tissue and the cardiac muscle. A characterization method was also developed to allow customization of important anatomical parameters in the DHS. The experimental results show that the DHS allowed highly realistic simulation of the beating heart, cardiac muscle, and pericardium under US imaging and has been demonstrated to enable successful US-guided pericardiocentesis.