Accurate geometric reconstruction of deformable tissues in monocular endoscopic video remains a fundamental challenge in robot-assisted minimally invasive surgery. Although recent volumetric and point primitive methods based on neural radiance fields (NeRF) and 3D Gaussian primitives have efficiently rendered surgical scenes, they still struggle with handling artifact-free tool occlusions and preserving fine anatomical details. These limitations stem from unrestricted Gaussian scaling and insufficient surface alignment constraints during reconstruction. To address these issues, we introduce Surgical Gaussian Surfels (SGS), which transform anisotropic point primitives into surface-aligned elliptical splats by constraining the scale component of the Gaussian covariance matrix along the view-aligned axis. We also introduce the Fully Fused Deformation Multilayer Perceptron (FFD-MLP), a lightweight Multi-Layer Perceptron (MLP) that predicts accurate surfel motion fields up to 5× faster than a standard MLP. This is coupled with locality constraints to handle complex tissue deformations. We use homodirectional view-space positional gradients to capture fine image details by splitting Gaussian Surfels in over-reconstructed regions. In addition, we define surface normals as the direction of the steepest density change within each Gaussian surfel primitive, enabling accurate normal estimation without requiring monocular normal priors. We evaluate our method on two in-vivo surgical datasets, where it outperforms current state-of-the-art methods in surface geometry, normal map quality, and rendering efficiency, while remaining competitive in real-time rendering performance. Code is available at https://git.new/NexyiHu.

Automating surgical systems enhances precision and safety while reducing human involvement in high-risk environments. A major challenge in automating surgical procedures like suturing is accurately modeling the suture thread, a highly flexible and compliant component. Existing models either lack the accuracy needed for safety-critical procedures or are too computationally intensive for real-time execution. In this work, we introduce a novel approach for modeling suture thread dynamics using control barrier functions (CBFs), achieving both realism and computational efficiency. Thread-like behavior, collision avoidance, stiffness, and damping are all modeled within a unified CBF and control Lyapunov function (CLFs) framework. Our approach eliminates the need to calculate complex forces or solve differential equations, significantly reducing computational overhead while maintaining a realistic model suitable for both automation and virtual reality surgical training systems. The framework also allows visual cues to be provided based on the thread's interaction with the environment, enhancing user experience when performing suture or ligation tasks. The proposed model is tested on the MagnetoSuture system, a minimally invasive robotic surgical platform that uses magnetic fields to manipulate suture needles, offering a less invasive solution for surgical procedures.

Surgical automation requires precise guidance and understanding of the scene. Current methods in the literature rely on bulky depth cameras to create maps of the anatomy; however, this does not translate well to space-limited clinical applications. Monocular cameras are small and allow minimally invasive surgeries in tight spaces, but additional processing is required to generate 3D scene understanding. We propose a 3D mapping pipeline that uses only RGB images to create segmented point clouds of the target anatomy. To ensure the most accurate reconstruction, we compare different structure from motion algorithms’ performance on mapping the central airway obstructions, and test the pipeline on a downstream task of tumor resection. In several metrics, including post-procedure percentage tissue charring, our pipeline performs comparably to RGB-D cameras and, in some cases, even surpasses their downstream task performance. These promising results demonstrate that automation guidance can be achieved in minimally invasive procedures with monocular cameras. This study is a step toward the complete autonomy of surgical robots.

Femoral artery access is essential for numerous clinical procedures, including diagnostic angiography, therapeutic catheterization, and emergency interventions. Despite its critical role, successful vascular access remains challenging due to anatomical variability, overlying adipose tissue, and the need for precise ultrasound (US) guidance. Needle placement errors can result in severe complications, thereby limiting the procedure to highly skilled clinicians operating in controlled hospital environments. While robotic systems have shown promise in addressing these challenges through autonomous scanning and vessel reconstruction, clinical translation remains limited due to reliance on simplified phantom models that fail to capture human anatomical complexity. In this work, we present a method for autonomous robotic US scanning of bifurcated femoral arteries, and validate it on five vascular phantoms created from real patient computed tomography (CT) data. Additionally, we introduce a video-based deep learning US segmentation network tailored for vascular imaging, enabling improved 3D arterial reconstruction. The proposed network achieves a Dice score of 89.21% and an Intersection over Union of 80.54% on a new vascular dataset. The reconstructed artery centerline is evaluated against ground truth CT data, showing an average L2error of 0.91±0.70 mm, with an average Hausdorff distance of 4.36±1.11mm. This study is the first to validate an autonomous robotic system for US scanning of the femoral artery on a diverse set of patient-specific phantoms, introducing a more advanced framework for evaluating robotic performance in vascular imaging and intervention.

Imitation learning (IL) has shown immense promise in enabling autonomous dexterous manipulations, including in learning surgical tasks. To fully unlock the potential of IL for surgery, access to clinical datasets is needed, which unfortunately lack the kinematic data required for current IL approaches. A promising source of large-scale surgical demonstrations is monocular surgical videos available online, making monocular pose estimation a crucial step toward enabling large-scale robot learning. Towards this end, we propose SurgiPose, a differentiable rendering-based approach to estimate kinematic information from monocular surgical videos, eliminating the need for direct access to ground-truth kinematics. Our method infers tool trajectories and joint angles by optimizing tool pose parameters to minimize the discrepancy between rendered and real images. To evaluate the effectiveness of our approach, we conduct experiments on two robotic surgical tasks—tissue lifting and needle pickup—using the da Vinci Research Kit Si (dVRK Si). We train imitation learning policies with both ground-truth measured kinematics and with estimated kinematics from video and compare their performance. Our results show that policies trained on estimated kinematics achieve comparable success rates to those trained on ground-truth data, demonstrating the feasibility of using monocular video-based kinematic estimation for surgical robot learning. By enabling kinematic estimation from monocular surgical videos, our work lays the foundation for large-scale learning of autonomous surgical policies from online surgical data.

Electrosurgery is a surgical technique that can improve tissue cutting by reducing cutting force and bleeding. However, electrosurgery adds a risk of thermal injury to surrounding tissue. Expert surgeons estimate desirable cutting velocities based on experience but have no quantifiable reference to indicate if a particular velocity is optimal. Furthermore, prior demonstrations of autonomous electrosurgery have primarily used constant tool velocity, which is not robust to changes in electrosurgical tissue characteristics, power settings, or tool type. Thermal imaging feedback provides information that can be used to reduce thermal injury while balancing cutting force by controlling tool velocity. We introduce Thermography for Electrosurgical Rate Modulation via Optimization (ThERMO) to autonomously reduce thermal injury while balancing cutting force by intelligently controlling tool velocity. We demonstrate ThERMO in tissue phantoms and compare its performance to the constant velocity approach. Overall, ThERMO improves cut success rate by a factor of three and can reduce peak cutting force by a factor of two. ThERMO responds to varying environmental disturbances, reduces damage to tissue, and completes cutting tasks that would otherwise result in catastrophic failure for the constant velocity approach.

Robotic suturing is a prototypical long-horizon dexterous manipulation task, requiring coordinated needle grasping, precise tissue penetration, and secure knot tying. Despite numerous efforts toward end-to-end autonomy, a fully autonomous suturing pipeline has yet to be demonstrated on physical hardware. We introduce SutureBot: an autonomous suturing benchmark on the da Vinci Research Kit (dVRK), spanning needle pickup, tissue insertion, and knot tying. To ensure repeatability, we release a high-fidelity dataset comprising 1,890 suturing demonstrations. Furthermore, we propose a goal-conditioned framework that explicitly optimizes insertion-point precision, improving targeting accuracy by 59\%-74\% over a task-only baseline. To establish this task as a benchmark for dexterous imitation learning, we evaluate state-of-the-art vision-language-action (VLA) models, including $\pi_0$, GR00T N1, OpenVLA-OFT, and multitask ACT, each augmented with a high-level task-prediction policy. Autonomous suturing is a key milestone toward achieving robotic autonomy in surgery. These contributions support reproducible evaluation and development of precision-focused, long-horizon dexterous manipulation policies necessary for end-to-end suturing. Dataset is available at: \href{https://huggingface.co/datasets/jchen396/suturebot}{Hugging Face}

Recent advances in robot-assisted surgery have resulted in progressively more precise, efficient, and minimally invasive procedures, sparking a new era of robotic surgical intervention. This enables doctors, in collaborative interaction with robots, to perform traditional or minimally invasive surgeries with improved outcomes through smaller incisions. Recent efforts are working toward making robotic surgery more autonomous which has the potential to reduce variability of surgical outcomes and reduce complication rates. Deep reinforcement learning methodologies offer scalable solutions for surgical automation, but their effectiveness relies on extensive data acquisition due to the absence of prior knowledge in successfully accomplishing tasks. Due to the intensive nature of simulated data collection, previous works have focused on making existing algorithms more efficient. In this work, we focus on making the simulator more efficient, making training data much more accessible than previously possible. We introduce Surgical Gym, an open-source high performance platform for surgical robot learning where both the physics simulation and reinforcement learning occur directly on the GPU. We demonstrate between 100-5000× faster training times compared with previous surgical learning platforms. The code is available at: https://github.com/SamuelSchmidgall/SurgicalGym.

Percutaneous needle insertions are commonly performed for diagnostic and therapeutic purposes as an effective alternative to more invasive surgical procedures. However, the outcome of needle-based approaches relies heavily on the accuracy of needle placement, which remains a challenge even with robot assistance and medical imaging guidance due to needle deflection caused by contact with soft tissues. In this paper, we present a novel mechanics-based 2D bevel-tip needle model that can account for the effect of nonlinear strain-dependent behavior of biological soft tissues under compression. Real-time finite element simulation allows multiple control inputs along the length of the needle with full three-degree-of-freedom (DOF) planar needle motions. Cross-validation studies using custom-designed multi-layer tissue phantoms as well as heterogeneous chicken breast tissues result in less than 1mm in-plane errors for insertions reaching depths of up to 61 mm, demonstrating the validity and generalizability of the proposed method.

In soft tissue surgeries, such as tumor resections, achieving precision is of utmost importance. Surgeons conventionally achieve this precision through intraoperative adjustments to the cutting plan, responding to deformations from tool-tissue interactions. This study examines the integration of physics-based tissue cutting simulations into autonomous robotic surgery to preoperatively predict and compensate for such deformations, aiming to improve surgical precision and reduce the necessity for dynamic adjustments during autonomous surgeries. This study adapts a real-to-sim-to-real workflow. Initially, the Autonomous System for Tumor Resection (ASTR) was employed to evaluate its accuracy in performing preoperatively intended incisions along the irregular contours of porcine tongue pseudotumors. Following this, a finite element analysis-based simulation, utilizing the Simulation Open Framework Architecture (SOFA), was developed and tuned to accurately mimic these tissue and incision interactions. Insights gained from this simulation were applied to refine the robot’s path planning, ensuring a closer alignment of actual incisions with the initially intended surgical plan. The efficacy of this approach was validated by comparing surface incision precision on ex vivo porcine tongues, with the average absolute error reducing from 1.73mm to 1.46mm after applying simulation-driven path adjustments (p < 0.001). Additionally, our method not only demonstrated improvements in maintaining the intended cutting shapes and locations, with shape matching scores using Hu moments enhancing from 0.10 to 0.06 and centroid shifts decreasing from 2.09mm to 1.33mm, but it also potentially reduced the likelihood of adverse oncologic outcomes by preventing clinically suggested excessively close margins of 2.2mm. This feasibility study suggests that merging physics-based cutting simulations with autonomous robotic surgery could potentially lead to more accurate incisions.

To track tumors during surgery, information from preoperative CT scans is used to determine their position. However, as the surgeon operates, the tumor may be deformed which presents a major hurdle for accurately resecting the tumor, and can lead to surgical inaccuracy, increased operation time, and excessive margins. This issue is particularly pronounced in robot-assisted partial nephrectomy (RAPN), where the kidney undergoes significant deformations during operation. Toward addressing this, we introduce a occupancy network-based method for the localization of tumors within kidney phantoms undergoing deformations at interactive speeds. We validate our method by introducing a 3D hydrogel kidney phantom embedded with exophytic and endophytic renal tumors. It closely mimics real tissue mechanics to simulate kidney deformation during in vivo surgery, providing excellent contrast and clear delineation of tumor margins to enable automatic threshold-based segmentation. Our findings indicate that the proposed method can localize tumors in moderately deforming kidneys with a margin of 6mm to 10mm, while providing essential volumetric 3D information at over 60Hz. This capability directly enables downstream tasks such as robotic resection.

This paper describes a novel tissue positioning system with an integrated suturing robot and demonstrates its ability to perform semi-automatic anastomoses of synthetic blood vessels. We began with a finite element analysis-based design consideration for achieving adequate grasping of blood vessels to demonstrate robust performance under expected clinical forces. We then conducted standardized positioning tests to measure the repeatability of the system and incorporated a high-resolution optical coherence tomography (OCT) fiber imaging sensor within the tip of the suturing tool to provide position feedback of the robot during a suturing task. Using the microvascular positioner and OCT sensor, the system performed semi-automatic suturing of synthetic 5 mm diameter blood vessels ($\mathrm{N}=4$), and the suture quality was evaluated for consistency in spacing, bite depth, percent lumen reduction, and maximum suture strength. The system completed the task in an average time of 31.75 minutes. The samples had zero missed stitches, average spacing of 1.64 mm, an average bite depth of 2.14 mm, an average lumen reduction of 57.98%, and an average suture strength of 3.13 N.

This article introduces a novel suture managing device (SMD) and new suture management controller to enable single-arm suture management during autonomous suturing with the Smart Tissue Autonomous Robot (STAR). The primary function of the SMD is to tension and manage the suture thread, a task that was previously carried out by a second manipulator or a human assistant. The SMD and its controller are integrated into STAR's autonomous suturing workflow. Experiments were conducted to quantify the tensioning force of SMD and to evaluate the suture quality of the new single-arm system. The prototype of SMD achieves 1.67N tensioning force with suturing time of 29.1±0.42 seconds per stitch. Our study results demonstrate that the single-arm STAR system with SMD achieves equivalent performance to our previous works in suturing efficiency where suture management was performed with either a dual-armed robotic system or by a human surgical assistant. The study's findings contribute to the field of medical robotics and to our knowledge represent the first known instance of single-arm suturing with suture management during autonomous anastomosis.

Surgical resection is the current clinical standard of care for treating squamous cell carcinoma. Maintaining an adequate tumor resection margin is the key to a good surgical outcome, but tumor edge delineation errors are inevitable with manual surgery due to difficulty in visualization and hand-eye coordination. Surgical automation is a growing field of robotics to relieve surgeon burdens and to achieve a consistent and potentially better surgical outcome. This paper reports a novel robotic supervised autonomous electrosurgery technique for soft tissue resection achieving millimeter accuracy. The tumor resection procedure is decomposed to the subtask level for a more direct understanding and automation. A 4-DOF suction system is developed, and integrated with a 6-DOF electrocautery robot to perform resection experiments. A novel near-infrared fluorescent marker is manually dispensed on cadaver samples to define a pseudotumor, and intraoperatively tracked using a dual-camera system. The autonomous dual-robot resection cooperation workflow is proposed and evaluated in this study. The integrated system achieves autonomous localization of the pseudotumor by tracking the near-infrared marker, and performs supervised autonomous resection in cadaver porcine tongues (N=3). The three pseudotumors were successfully removed from porcine samples. The evaluated average surface and depth resection errors are 1.19 and 1.83mm, respectively. This work is an essential step towards autonomous tumor resections.

Vertebral compression fractures (VCFs), the most common fragility fractures secondary to osteoporosis, affect more than 200 million individuals worldwide. Percutaneous vertebral augmentation is an effective interventional treatment option that is routinely performed across the world. Because fluoroscopy-guided vertebral augmentation is a well-established and safe minimally invasive technique, automating its delivery is among the most important next steps. In this work, we describe the design and evaluation of a novel cannula mounted vertebral augmentation robot in a simulated X-ray environment as a first step toward autonomous vertebral augmentation. The cannula robot employs a piezo stack with inchworm control to place surgical tools within the vertebral body, while X-ray imaging verifies the robot does not interfere with imaging. Finite element analysis of the robot confirms that radiolucent materials were rigid enough to be used in the robot design as expected deformations for the cannula drive, accessory drive, and locking mechanisms$(1.299 \pm 0.034 \ um, 1.280 \pm 0.027\ um$, and$1.960 \pm 0.218\ um$, respectively) did not exceed the stroke lengths of the piezo stacks. An in silico clinical trial based on a human anatomy model suffering from VCF validates that the cannula robot does not impede visualization of the critical anatomy and tool-to-tissue positioning. Together these results demonstrate the feasibility of a cannula mounted robot for vertebral augmentation.

Exerting forces and torques instantaneously on rigid magnetic bodies with no physical connection is an attractive feature of magnetic robotics. This demonstrates great potential for manipulating tools that are externally controlled through the use of magnetic fields in minimally invasive surgeries. The magnetic field can be controlled by the application of currents to electromagnets positioned around the surgical site, and the necessary currents for a specific desired manipulation can be derived from magnetic field models. However, the magnetic field generated by electromagnetic coils are highly nonlinear, especially in the vicinity of the magnetic field sources, which complicates the modeling process. While simple dipole models provide a good approximation for these fields far away from the electromagnets, these models tend to be highly inaccurate near the sources. Magnetic surgical applications benefit from models which accurately describe fields and gradients both near and far from the field source. Particularly, since forces and torques decay inversely proportionally with the cube of the distance to the coil, inaccurate modeling near the coil makes large regions near the coil unfit for applications requiring precisely predicted motion. Estimation errors near coils generate inaccuracies in field models that significantly reduce control performance for rigid magnetic bodies. In order to tackle this problem, we utilize Zernike basis functions to analytically represent the nonlinear magnetic field distribution more accurately. The accuracy of the controller is tested experimentally by driving a magnetic surgical suture needle with a length of 22 mm in the MagnetoSuture™ system along a lemniscate trajectory. The magnetic needle's tip position and the needle orientation, autonomously controlled by the proposed controller, shows RMS tracking error of 2.35 mm using typical dipole models and 1.71 mm for the Zernike fitting approach, a 27% improvement in tracking error. This suggests that the use of Zernike basis functions to capture the nonlinearities of the magnetic field may assist in implementing fast and precise autonomous control strategies for magnetic suture needles.

Autonomous robotic suturing has the potential to improve surgery outcomes by leveraging accuracy, repeatability, and consistency compared to manual operations. However, achieving full autonomy in complex surgical environments is not practical and human supervision is required to guarantee safety. In this paper, we develop a confidence-based supervised autonomous suturing method to perform robotic suturing tasks via both Smart Tissue Autonomous Robot (STAR) and surgeon collaboratively with the highest possible degree of autonomy. Via the proposed method, STAR performs autonomous suturing when highly confident and otherwise asks the operator for possible assistance in suture positioning adjustments. We evaluate the accuracy of our proposed control method via robotic suturing tests on synthetic vaginal cuff tissues and compare them to the results of vaginal cuff closures performed by an experienced surgeon. Our test results indicate that by using the proposed confidence-based method, STAR can predict the success of pure autonomous suture placement with an accuracy of 94.74%. Moreover, via an additional 25% human intervention, STAR can achieve a 98.1% suture placement accuracy compared to an 85.4% accuracy of completely autonomous robotic suturing. Finally, our experiment results indicate that STAR using the proposed method achieves 1.6 times better consistency in suture spacing and 1.8 times better consistency in suture bite sizes than the manual results.

This paper reports the design and evaluation of a novel piezo based actuator for needle drive in autonomous Deep Anterior Lamellar Keratoplasty (piezo-DALK). The actuator weighs less than 8g and is 20mm × 20mm × 10.5mm in size, making it ideal for eye-mounted applications. Mean open loop positional deviation was 1.17 ± 3.15um, and system repeatability and accuracy were 17.16um and 18.33um, respectively. Stall force was found to vary linearly with the cooling cycle and the actuator achieved a maximum drive force of 3.98N. When simulating the DALK procedure in synthetic corneal tissue, the piezo-DALK achieved a penetration depth of 643.56um which was equivalent to 92.1% of the total corneal thickness. This correlated closely with our desired depth of 90% ± 5% and took 2.5 hours to achieve. This work represents the first eye mountable actuator capable of "Big Bubble" needle drive for autonomous DALK procedures.

Real-time visual localization of needles is necessary for various surgical applications, including surgical automation and visual feedback. In this study we investigate localization and autonomous robotic control of needles in the context of our magneto-suturing system. Our system holds the potential for surgical manipulation with the benefit of minimal invasiveness and reduced patient side effects. However, the nonlinear magnetic fields produce unintuitive forces and demand delicate position-based control that exceeds the capabilities of direct human manipulation. This makes automatic needle localization a necessity. Our localization method combines neural network-based segmentation and classical techniques, and we are able to consistently locate our needle with 0.73 mm RMS error in clean environments and 2.72 mm RMS error in challenging environments with blood and occlusion. The average localization RMS error is 2.16 mm for all environments we used in the experiments. We combine this localization method with our closed-loop feedback control system to demonstrate the further applicability of localization to autonomous control. Our needle is able to follow a running suture path in (1) no blood, no tissue; (2) heavy blood, no tissue; (3) no blood, with tissue; and (4) heavy blood, with tissue environments. The tip position tracking error ranges from 2.6 mm to 3.7 mm RMS, opening the door towards autonomous suturing tasks.

This paper proposes a magnetic needle steering controller to manipulate mesoscale magnetic suture needles for executing planned suturing motion. This is an initial step towards our research objective: enabling autonomous control of magnetic suture needles for suturing tasks in minimally invasive surgery. To demonstrate the feasibility of accurate motion control, we employ a cardinally-arranged four-coil electromagnetic system setup and control magnetic suture needles in a 2-dimensional environment, i.e., a Petri dish filled with viscous liquid. Different from only using magnetic field gradients to control small magnetic agents under high damping conditions, the dynamics of a magnetic suture needle are investigated and encoded in the controller. Based on mathematical formulations of magnetic force and torque applied on the needle, we develop a kinematically constrained dynamic model that controls the needle to rotate and only translate along its central axis for mimicking the behavior of surgical sutures. A current controller of the electromagnetic system combining with closed-loop control schemes is designed for commanding the magnetic suture needles to achieve desired linear and angular velocities. To evaluate control performance of magnetic suture needles, we conduct experiments including needle rotation control, needle position control by using discretized trajectories, and velocity control by using a time-varying circular trajectory. The experiment results demonstrate our proposed needle steering controller can perform accurate motion control of mesoscale magnetic suture needles.

Patients with single ventricle heart defect undergo Fontan surgery to reroute the blood flow from the lower body to the lung by connecting the inferior vena cava to the pulmonary artery using a vascular graft. Since each patient has an unique anatomical structure and blood flow dynamics, the graft design is a critical factor for maximizing the long-term survival rate of Fontan patients. Currently, designing and evaluating grafts involve computer aided design (CAD) and computational fluid dynamics (CFD) skills. CAD incorporates numerous tools for design but lacks depth perception, surgical features, and design parameters for creating vascular grafts while visualizing and modifying patient anatomies. These limitations may lead to long lead times, inconsistent workflow, and surgically infeasible graft designs. In this paper, we introduce a novel virtual reality vascular graft modeling software - CorFix, that provides solutions to these challenges. CorFix includes several visualization features for performing diagnostics and surgical features with design guidelines for creating patient specific tube-shaped grafts in 3D. The designed vascular graft can be exported into a 3D model, which can be utilized for performing computational fluid dynamic analysis and 3D printing. The patient specific vascular graft designs in CorFix were compared to an engineering CAD software, SolidWorks (Dassault Systèmes, Vélizy-Villacoublay, France), by 8 participants. Through all participants had only received one time 10-minute tutorial on CorFix, CorFix had a higher success rate and 3.4 times faster performance in designing surgically feasible grafts than CAD. CorFix also scored higher in usability and lower in perceived workload than CAD. CorFix may be the tool that can enable medical doctors without 3D modeling background to design patient specific grafts.

Patient-specific biodegradable grafts target to enhance surgical repairs of complex congenital heart defects (CHD). This study reports the design, simulation, and creation of bifurcated right ventricle-pulmonary artery (RVPA) conduit grafts for patients with CHD. The original right ventricle outflow tract and RVPA conduit-anatomies of two patients (n=2) who previously underwent Rastelli type surgical repair for their CHD were created using medical image segmentation software based on magnetic resonance imaging data. The pulsatile RVPA flow was simulated utilizing computational fluid dynamics (CFD) to calculate important hemodynamic parameters. The re-designed RVPA geometries for the patients were created by varying the radius and angle of the pulmonary artery bifurcation. The wall shear stress and power loss results of the re-designed RVPA models were compared to identify the best performing graft. The hemodynamic results demonstrated that the designed optimized grafts outperformed the original grafts. To test the feasibility of designed grafts in vivo, the bifurcated RVPA conduit of a pig was manufactured using a 3D printed mandrel and electrospinning technique before the implantation. The implanted graft allowed new tissue formation within weeks. The results of our study and simulations provide an insight into the creation of optimal performing tissue-engineered bifurcated grafts for the patients with CHD in the surgical planning process. Integration of flow simulations to support design and electrospinning technique to manufacture patient-specific biodegradable grafts has the potential to improve surgical outcomes in CHD.

Trauma is among the leading causes of death in the United States with up to 29% of pre-hospital trauma deaths attributed to uncontrolled hemorrhages. This paper reports a semi-autonomous robotic system capable of assessing trauma using 2D and 3D image analysis and enabling remote focused assessment with sonography for trauma (FAST) en route to the hospital for earlier trauma diagnosis and faster initialization of life saving care. The system was able to accurately calculate FAST scan positions of patient specific phantoms using the measured phantom sizes and positions of the umbilicus. The system was capable of accurately classifying and localizing wounds, so they can be avoided during the ultrasound scan. These objects were localized with an accuracy of 0.94 ± 0.179cm and FAST exam locations were estimated with an accuracy of 2.2 ± 1.88cm. A radiologist successfully completed a remote FAST scan of the phantom using the system with improved image quality over manual scans, demonstrating feasibility of the system.

Skin segmentation plays an important role in a wide variety of biomedical image processing applications, such as skin cancer identification, skin lesion detection, and wound isolation. However, contemporary research has been mainly based on facial and hand skin datasets, with no other body regions considered for skin pixels sampling. Segmenting skin specifically in the abdominal region can aid in robotic abdominal surgeries and treatment procedures, such as robot-assisted laparoscopic surgeries and abdominal ultrasounds. A robust and highly accurate abdominal skin detection technique thus becomes imperative. To this end, we compiled a novel dataset of 1,400 segmented abdominal pictures and adapted and compared four abdominal skin segmentation techniques: one based on thresholding and three deep learning techniques, namely a fully connected neural network for pixel-level classification, and two convolution-based networks, U-Net and Mask-RCNN. We show that the U-Net model outperforms the other segmentation techniques, resulting in a pixel-to-pixel mean cross-validation accuracy of 95.51% on our Abdominal dataset. The incorporation of the Abdominal dataset in the training helped improve the abdominal skin segmentation accuracy by 10.19%. The U-Net model proved to be computationally the fastest, enabling real time skin segmentation with a processing rate of 37 frames per second.

Compared to open surgical techniques, laparoscopic surgical methods aim to reduce the collateral tissue damage and hence decrease the patient recovery time. However, constraints imposed by the laparoscopic surgery, i.e. the operation of surgical tools in limited spaces, turn simple surgical tasks such as suturing into time-consuming and inconsistent tasks for surgeons. In this paper, we develop an autonomous laparoscopic robotic suturing system. More specific, we expand our smart tissue anastomosis robot (STAR) by developing i) a new 3D imaging endoscope, ii) a novel actuated laparoscopic suturing tool, and iii) a suture planning strategy for the autonomous suturing. We experimentally test the accuracy and consistency of our developed system and compare it to sutures performed manually by surgeons. Our test results on suture pads indicate that STAR can reach 2.9 times better consistency in suture spacing compared to manual method and also eliminate suture repositioning and adjustments. Moreover, the consistency of suture bite sizes obtained by STAR matches with those obtained by manual suturing.

This paper reports the design and evaluation of a novel robotic device for cornea transplant surgery. The device enables the OCT-sensor guided Big Bubble hydro-dissection approach for deep anterior lamellar keratoplasty (DALK) cornea transplant surgery. DALK is highly challenging because it requires precise placement of a needle into the stroma of the cornea down to Descemets Membrane (DM) and injects a fluid to separate the remaining stroma from Descemet's membrane. Finally, the stroma is removed and replaced with the donor cornea graft. Compared to traditional penetrating keratoplasty (PK), which involves a full-thickness graft, this method significantly reduces the risk of rejection of the donor cornea by keeping the DM intact. A comparison of autonomous OCT guided needle insertions with expert manual needle insertions showed that the device significantly increased the precision and consistency of the needle placement, which could lead to better visual outcomes and fewer complications. In a study on cadaver porcine eyes, the measured insertion depth as a percentage of cornea thickness for the robotic device was 90.05% +/- 2.33% compared to 79.16% +/- 5.68% for manual insertions.

This paper reports a robotic laparoscopic surgery system performing electro-surgery on porcine cadaver kidney, and evaluates its accuracy in an open loop control scheme to conduct targeting and cutting tasks guided by a novel 3D endoscope. We describe the design and integration of the novel laparoscopic imaging system that is capable of reconstructing the surgical field using structured light. A targeting task is first performed to determine the average positioning error of the system as guided by the laparoscopic camera. The imaging system is then used to reconstruct the surface of a porcine cadaver kidney, and generate a cutting trajectory with consistent depth. The paper concludes by using the robotic system in open loop control to cut this trajectory using a multi degree of freedom electro-surgical tool. It is demonstrated that for a cutting depth of 3 mm, the robotic surgical system follows the trajectory with an average depth of 2.44 mm and standard deviation of 0.34 mm. The average positional accuracy of the system was 2.74±0.99 mm.

Autonomous robotic assisted surgery (RAS) systems aim to reduce human errors and improve patient outcomes leveraging robotic accuracy and repeatability during surgical procedures. However, full automation of RAS in complex surgical environments is still not feasible and collaboration with the surgeon is required for safe and effective use. In this work, we utilize our Smart Tissue Autonomous Robot (STAR) to develop and evaluate a shared control strategy for the collaboration of the robot with a human operator in surgical scenarios. We consider 2D pattern cutting tasks with partial blood occlusion of the cutting pattern using a robotic electrocautery tool. For this surgical task and RAS system, we i) develop a confidence-based shared control strategy, ii) assess the pattern tracking performances of manual and autonomous controls and identify the confidence models for human and robot as well as a confidence-based control allocation function, and iii) experimentally evaluate the accuracy of our proposed shared control strategy. In our experiments on porcine fat samples, by combining the best elements of autonomous robot controller with complementary skills of a human operator, our proposed control strategy improved the cutting accuracy by 6.4%, while reducing the operator work time to 44% compared to a pure manual control.

This paper specifies a surgical robot performing semi-autonomous electrosurgery for tumor resection and evaluates its accuracy using a visual servoing paradigm. We describe the design and integration of a novel, multi-degree of freedom electrosurgical tool for the smart tissue autonomous robot (STAR). Standardized line tests are executed to determine ideal cut parameters in three different types of porcine tissue. STAR is then programmed with the ideal cut setting for porcine tissue and compared against expert surgeons using open and laparoscopic techniques in a line cutting task. We conclude with a proof of concept demonstration using STAR to semi-autonomously resect pseudo-tumors in porcine tissue using visual servoing. When tasked to excise tumors with a consistent 4mm margin, STAR can semi-autonomously dissect tissue with an average margin of 3.67 mm and a standard deviation of 0.89mm.

Three-dimensional sensing of changing surgical scenes would improve the function of surgical robots. This paper explores the requirements and utility of a new type of depth sensor, the plenoptic camera, for surgical robots. We present a metric calibration procedure for the plenoptic camera and the registration of its coordinate frame to the robot (hand-eye calibration). We also demonstrate the utility in robotic needle insertion and application of sutures in phantoms. The metric calibration accuracy is reported as 1.14 ± 0.80 mm for the plenoptic camera and 1.57 ± 0.90 mm for hand-eye calibration. The accuracy of needle insertion task is 1.79 ± 0.35 mm for the entire robotic system. Additionally, the accuracy of suture placement with the presented system is reported at 1.80 ± 0.43 mm. Finally, we report consistent suture spacing with only 0.11 mm standard deviation between inter-suture distances. The measured accuracy of less than 2 mm with consistent suture spacing is a promising result to provide repeatable leak-free suturing with a robotic tool and a plenoptic depth imager.

This paper reports an evaluation of contact-less hand tracking sensors for the use of tele-operation, in particular for surgical robotics applications. Two hand tracking systems are investigated: 3Gear Systems interface with the Microsoft KinectTMsensor, and the Leap Motion sensor system. This paper reports an experimental evaluation and comparison of the two systems range, static positioning error, trajectory accuracy of single finger and hand motions, and latency. Latency and trajectory accuracy were found superior using the Leap system. KinectTM/3Gear was found superior when larger range and gesture control are necessary. 3Gear was used in a simulated surgical positioning task and demonstrated an average translational accuracy of 6.2mm. Given the data we have collected, we conclude that neither system, at present, possesses the high level of accuracy and robustness over the required range that would be a prerequisite for use as a medical robotics master.

This paper specifies and evaluates the accuracy of the Smart Tissue Anastomosis Robot (STAR). The STAR is a proof of concept vision-guided robotic system equipped with an actuated laparoscopic suturing tool and a multispectral vision system. The STAR supports image-based suturing commands and is capable of detecting near-infrared fluorescent (NIRF) markers that provide reliable visual segmentation and tracking. The paper reports the best case scenario accuracy specifications of the STAR as derived from its configuration and calibration parameters. We also evaluate experimentally the effects of overlaying NIRF markers on the accuracy of the STAR when these markers are used as the source of image-based commands and we compare these results to the accuracy of the STAR with image-based commands generated from plain color images. Our results demonstrate that the STAR is able to place sutures on a planar phantom with an average accuracy of 0.5 mm with a standard deviation of 0.2 mm and that NIRF markers have no statistically significant adverse effect on the accuracy.

This paper reports the design, development, and magnetic resonance imaging (MRI) compatibility evaluation of an actuated transrectal prostate robot for MRI-guided intervention. The robot employs an actuated needle guide with the goal of reducing interventional procedure times and increasing needle placement accuracy. The design of the robot, employing piezo-ceramic-motor actuated needle guide positioning and manual needle insertion, is reported. Results of a MRI compatibility study show no reduction of MRI image signal-to-noise-ratio (SNR) with the motors disabled and a 40% to 60% reduction in SNR with the motors enabled. The addition of radio-frequency (RF) shielding is shown to significantly reduce image SNR degradation due to the presence of the robotic device.

This paper reports a novel hybrid method of tracking the position and orientation of robotic medical instruments within the imaging volume of a magnetic resonance imaging (MRI) system. The method utilizes two complementary measurement techniques: passive MRI fiducial markers and MRI compatible joint encoding. This paper reports an experimental evaluation of the tracking accuracy of this system. The accuracy of this system compares favorably to that of a previously reported active tracking system. Moreover, the hybrid system is quickly and easily deployed on different MRI scanner systems

This work reports a novel remotely actuated manipulator for transrectal prostate imaging and intervention, designed for use in a standard cylindrical, high-field magnetic resonance imaging (MRI) scanner. The device provides three-dimensional MRI guided needle placement with millimeter accuracy under physician control. Candidate procedures enabled by this device include MRI guided needle biopsy, fiducial marker placements, and therapy delivery. Its compact size allows for use in both standard cylindrical and open configuration MRI scanners. Preliminary in-vivo canine experiments are reported.