Augmented Reality (AR) has been proven beneficial to External Ventricular Drain (EVD) surgery by providing in-situ visual guidance during operations. During this procedure, the key challenge is estimating the spatial relationship between pre-operative images and actual patient anatomy accurately and efficiently. Previous works have revealed conflicts between tracking accuracy, workflow efficiency, and non-invasiveness in tracking pipelines. This research fully utilizes the capabilities of Time of Flight (ToF) depth sensors, including retro-reflective tool tracking and dense surface information, to construct a convenient and accurate EVD guiding pipeline. As previous studies have proven significant depth errors in ToF depth sensors, we first evaluated the feasibility of using ToF sensors in surgical guidance by estimating its accuracy under different conditions and corrected this error in our pipeline. Our results show $ \text{7.580}\pm \text{1.488}\,\text{mm}$7.580±1.488mm depth value errors on human skin under HoloLens 2 depth camera, indicating the significance of depth correction. This error was reduced by over 85% using proposed depth correction method on head phantoms in different materials. The corrected depth information can then be utilized to reconstruct the head surface with sub-millimeter accuracy, validated on a series of 3D-printed models and a sheep head. To demonstrate the effectiveness of the proposed framework, we conducted a case study simulating EVD surgery. Five surgeons were involved in this study, each performing nine k-wire insertions on a head phantom under virtual guidance without tracking for surgical tools. The results revealed $ \text{2.09} \pm \text{1.00}\,\text{mm}$2.09±1.00mm translational and $\text{2.97}\pm \text{1.95}^\circ$2.97±1.95∘ orientational guidance accuracy, demonstrating competitive performance with previous research.

Ankle fractures are one of the most common types of fractures in clinical practice, and incorrect assessment of joint stability during surgery can easily lead to various ligament injuries and complications such as arthritis. Ultrasonic imaging is currently one of the least invasive and least burdensome methods for diagnosing ankle fractures in patients. However, previous studies mostly diagnosed joint fractures by segmenting the shape and position of the entire bone, while the latest clinical research found that joint stability can be determined by measuring the gap between two segments of bone. Therefore, we propose a new model that accurately detects skeletal edge points on both sides of the ankle joint, measures the length of the bone gap, and thus assists doctors in accurate diagnosis or treatment planning. Our model uses a Region Proposal Network and Transformer encoder as the basic backbone. It first filters the background to obtain rough interesting regions, then collectively models the internal attention between background pixels and target pixels to enhance the distinction between the skeletal main body and background. Additionally, it extends frequency domain recognition to strengthen the differentiation between skeletal main body and background, thereby promoting edge point detection. We use ankle joint ultrasonic images from real patients, annotated by bone disease experts. Experimental results demonstrate that our model can accurately locate the detailed positions of skeletal edge points, even in images containing complex body tissues, ensuring correct detection. Furthermore, In addition, our model can process 24 frames per second and has the ability to track frames in a video.

During traditional surgeries, planning and instrument guidance is displayed on an external screen. Recent developments of augmented reality (AR) techniques can overcome obstacles including hand-eye discoordination and heavy mental load. Among these AR technologies, optical see-through (OST) schemes with stereoscopic displays can provide depth perception and retain the physical scene for safety considerations. However, limitations still exist in certain AR systems and the influence of these factors on surgical performance is yet to explore. To this end, experiments of multi-scale surgical tasks were carried out to compare head-mounted display (HMD) AR and autostereoscopic image overlay (AIO) AR, concerning objective performance and subjective evaluation. To solely analyze effects brought by display techniques, the tracking system in each included display system was identical and similar tracking accuracy was proved by a preliminary experiment. Focus and context rendering was utilized to enhance in-situ visualization for surgical guidance. Latency values of all display systems were assessed and a delay experiment proved the latency differences had no significant impact on user performance. Results of multi-scale surgical tasks showed that HMD outperformed in detailed operations probably due to stable resolution along the depth axis, while AIO had better performance in larger-scale operations for better depth perception. This article helps point out the critical limitations of current OST AR techniques and potentially promotes the progress of AR applications in surgical guidance.

The use of Augmented Reality (AR) for navigation purposes has shown beneficial in assisting physicians during the performance of surgical procedures. These applications commonly require knowing the pose of surgical tools and patients to provide visual information that surgeons can use during the performance of the task. Existing medical-grade tracking systems use infrared cameras placed inside the Operating Room (OR) to identify retro-reflective markers attached to objects of interest and compute their pose. Some commercially available AR Head-Mounted Displays (HMDs) use similar cameras for self-localization, hand tracking, and estimating the objects' depth. This work presents a framework that uses the built-in cameras of AR HMDs to enable accurate tracking of retro-reflective markers without the need to integrate any additional electronics into the HMD. The proposed framework can simultaneously track multiple tools without having previous knowledge of their geometry and only requires establishing a local network between the headset and a workstation. Our results show that the tracking and detection of the markers can be achieved with an accuracy of$0.09\pm 0.06\ mm$on lateral translation,$0.42 \pm 0.32\ mm$on longitudinal translation and$0.80 \pm 0.39^\circ$for rotations around the vertical axis. Furthermore, to showcase the relevance of the proposed framework, we evaluate the system's performance in the context of surgical procedures. This use case was designed to replicate the scenarios of k-wire insertions in orthopedic procedures. For evaluation, seven surgeons were provided with visual navigation and asked to perform 24 injections using the proposed framework. A second study with ten participants served to investigate the capabilities of the framework in the context of more general scenarios. Results from these studies provided comparable accuracy to those reported in the literature for AR-based navigation procedures.

Automatic femur segmentation from computed tomography volume is a crucial but challenging task for computer-aided diagnosis in orthopedic surgeries. The main obstacles are weak bone boundaries, narrowness of joint space, variations in femur density and shape, as well as diverse leg postures. In this paper, we presented a novel 3-D feature-enhanced network to address these challenges. The novelty of our approach lies in two feature enhancement modules, including the edge detection task and the multi-scale features fusion. First, the edge detection task was embedded into femur segmentation from computed tomography volume to solve the problems of narrow joint space and weak femur boundary. Crucially, a task-specific edge detector was used to optimize the performance of femur segmentation in an end-to-end trainable system. Second, the multi-scale features fusion provided both local and global contexts to handle the problems of large variations in leg postures as well as femur shape and density. The results demonstrated that accurate 3-D femur segmentation with a high Dice similarity coefficient of 96.88% was achieved using the developed method, and the segmentation of computed tomography volume took 0.93 s on an average.

Intramedullary (IM) nail implantation is currently the standard treatment for femoral intertrochanteric fractures. However, individual differences in femur cavity bring a challenge in designing well-matched IM nails and cause difficulties in IM nail implantation. Therefore, there is an intense need to analyze femur cavities to predict difficulties in IM nail implantation to assist the design of IM nails. This study proposed a method to automatically identify subtypes of femur cavities that exhibit differences in potential difficulties in nail implantation by clustering the morphological features of femur models. The unsupervised subtype extraction method offers a scientific approach to stratify patients for designing and choosing well-matched IM nails. First, the quantitative morphological features of 422 femur cavities were extracted from computed tomography patient models. Second, 422 femur cavities were clustered into three distinct subtypes using a density peak-based k-means clustering method to provide a possible solution for the scientific design of IM nails. The effectiveness of the identified subtypes was validated by comparing subtype differences associated with IM nail implantation and the natural attributes of the patient. Quantitative evaluation of the mismatch degree and real clinical cases confirmed that the clustering results were clinically effective, with clear differences in the subtypes. Therefore, particular IM nails designed from the identified subtypes will potentially facilitate IM nail implantation and reduce complications. Compared with state-of-the-art methods, we used the largest scale dataset and unsupervised clustering to achieve subtype identification of femur cavities with clinical significance.