Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of target anatomy, surrounding tissue, and instrumentation, but there are significant challenges in harnessing it for effectively guiding interventional procedures. Challenges include the strong static magnetic field, rapidly switching magnetic field gradients, high-power radio frequency pulses, sensitivity to electrical noise, and constrained space to operate within the bore of the scanner. MRI has a number of advantages over other medical imaging modalities, including no ionizing radiation, excellent soft-tissue contrast that allows for visualization of tumors and other features that are not readily visible by other modalities, true 3-D imaging capabilities, including the ability to image arbitrary scan plane geometry or perform volumetric imaging, and capability for multimodality sensing, including diffusion, dynamic contrast, blood flow, blood oxygenation, temperature, and tracking of biomarkers. The use of robotic assistants within the MRI bore, alongside the patient during imaging, enables intraoperative MR imaging (iMRI) to guide a surgical intervention in a closed-loop fashion that can include tracking of tissue deformation and target motion, localization of instrumentation, and monitoring of therapy delivery. With the ever-expanding clinical use of MRI, MRI-compatible robotic systems have been heralded as a new approach to assist interventional procedures to allow physicians to treat patients more accurately and effectively. Deploying robotic systems inside the bore synergizes the visual capability of MRI and the manipulation capability of robotic assistance, resulting in a closed-loop surgery architecture. This article details the challenges and history of robotic systems intended to operate in an MRI environment and outlines promising clinical applications and associated state-of-the-art MRI-compatible robotic systems and technology for making this possible.

Lumbar injection is an image-guided procedure performed manually for diagnosis and treatment of lower back pain and leg pain. Previously, we have developed and verified an MR-Conditional robotic solution to assisting the needle insertion process. Drawing on our clinical experiences, a virtual remote center of motion (RCM) constraint is implemented to enable our robot to mimic a clinician’s hand motion to adjust the needle tip position in situ. Force and image data are collected to study the needle behavior in gel phantoms during this motion, and a mechanics-based needle-tissue interaction model is proposed and evaluated to further examine the underlying physics. This work extends the commonly-adopted notion of an RCM for flexible needles, and introduces new motion parameters to describe the needle behavior. The model parameters can be tuned to match the experimental result to sub-millimeter accuracy, and this proposed needle manipulation method presents a safer alternative to laterally translating the needle during in situ needle adjustments.

This paper reports the development of a fully actuated body-mounted robotic assistant for MRI-guided low back pain injection. The robot is designed with a 4-DOF needle alignment module and a 2-DOF remotely actuated needle driver module. The 6-DOF fully actuated robot can operate inside the scanner bore during imaging; hence, minimizing the need of moving the patient in or out of the scanner during the procedure, and thus potentially reducing the procedure time and streamlining the workflow. The robot is built with a lightweight and compact structure that can be attached directly to the patient's lower back using straps; therefore, attenuating the effect of patient motion by moving with the patient. The novel remote actuation design of the needle driver module with beaded chain transmission can reduce the weight and profile on the patient, as well as minimize the imaging degradation caused by the actuation electronics. The free space positioning accuracy of the system was evaluated with an optical tracking system, demonstrating the mean absolute errors (MAE) of the tip position to be 0.99±0.46 mm and orientation to be 0.99±0.65°. Qualitative imaging quality evaluation was performed on a human volunteer, revealing minimal visible image degradation that should not affect the procedure. The mounting stability of the system was assessed on a human volunteer, indicating the 3D position variation of target movement with respect to the robot frame to be less than 0.7 mm.

This paper presents a body-mounted, four degree-of-freedom (4-DOF) parallel mechanism robot for image-guided percutaneous interventions. The design of the robot is optimized to be light weight and compact such that it could be mounted to the patient body. It has a modular design that can be adopted for assisting various image-guided, needle-based percutaneous interventions such as arthrography, biopsy and brachytherapy seed placement. The robot mechanism and the control system are designed and manufactured with components compatible with imaging modalities including Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). The current version of the robot presented in this paper is optimized for shoulder arthrography under MRI guidance; a Z-shaped fiducial frame is attached to the robot, providing accurate and repeatable robot registration with the MR scanner coordinate system. Here we present the mechanical design of the manipulator, robot kinematics, robot calibration procedure, and preliminary bench-top accuracy assessment. The bench-top accuracy evaluation of the robotic manipulator shows average translational error of 1.01 mm and 0.96 mm in X and Z axes, respectively, and average rotational error of 3.06 degrees and 2.07 degrees about the X and Z axes, respectively.

Flexible ureteroscopes are widely used for inspection, access, and manipulation in the ureter and kidney during uretero-renoscopy. However, existing endoscope designs rely on decades-old manual controls for translation, rotation, and tip flexion. The development of a more intuitive and user-friendly control system has the potential to greatly enhance the safety, efficacy, and efficiency of endoscopic procedures. Based on a first generation prototype, our lab designed an ergonomically enhanced flexible endoscopy robotic system. The system was designed to enable the surgeon to operate the endoscope through a low cost game controller. The kinematics were analyzed considering the nonlinearity of flexion driving and cable slack. The kinematics control was combined with the direct control for the system. The initial evaluations show the feasibility of this approach.

This article addresses the two key challenges in computer-assisted percutaneous tumor ablation: planning multiple overlapping ablations for large tumors while avoiding critical structures, and executing the prescribed plan. Toward semiautomatic treatment planning for image-guided surgical interventions, we develop a systematic approach to the needle-based ablation placement task, ranging from preoperative planning algorithms to an intraoperative execution platform. The planning system incorporates clinical constraints on ablations and trajectories using a multiple objective optimization formulation, which consists of optimal path selection and ablation coverage optimization based on integer programming. The system implementation is presented and validated in both phantom and animal studies. The presented system can potentially be further extended for other ablation techniques such as cryotherapy.

Object tracking is a key enabling technology in the context of computer-assisted medical interventions. Allowing the continuous localization of medical instruments and patient anatomy, it is a prerequisite for providing instrument guidance to subsurface anatomical structures. The only widely used technique that enables real-time tracking of small objects without line-of-sight restrictions is electromagnetic (EM) tracking. While EM tracking has been the subject of many research efforts, clinical applications have been slow to emerge. The aim of this review paper is therefore to provide insight into the future potential and limitations of EM tracking for medical use. We describe the basic working principles of EM tracking systems, list the main sources of error, and summarize the published studies on tracking accuracy, precision and robustness along with the corresponding validation protocols proposed. State-of-the-art approaches to error compensation are also reviewed in depth. Finally, an overview of the clinical applications addressed with EM tracking is given. Throughout the paper, we report not only on scientific progress, but also provide a review on commercial systems. Given the continuous debate on the applicability of EM tracking in medicine, this paper provides a timely overview of the state-of-the-art in the field.

The introduction of software technology in a life-dependent environment requires the development team to execute a process that ensures a high level of software reliability and correctness. Despite their popularity, agile methods are generally assumed to be inappropriate as a process family in these environments due to their lack of emphasis on documentation, traceability, and other formal techniques. Agile methods, notably Scrum, favor empirical process control, or small constant adjustments in a tight feedback loop. This paper challenges the assumption that agile methods are inappropriate for safety-critical software development. Agile methods are flexible enough to encourage the rightamount of ceremony; therefore if safety-critical systems require greater emphasis on activities like formal specification and requirements management, then an agile process will include these as necessary activities. Furthermore, agile methods focus more on continuous process management and code-level quality than classic software engineering process models. We present our experiences on the image-guided surgical toolkit (IGSTK) project as a backdrop. IGSTK is an open source software project employing agile practices since 2004. We started with the assumption that a lighter process is better, focused on evolving code, and only adding process elements as the need arose. IGSTK has been adopted by teaching hospitals and research labs, and used for clinical trials. Agile methods have matured since the academic community suggested they are not suitable for safety-critical systems almost a decade ago, we present our experiences as a case study for renewing the discussion.

Needle steering is a commonly used technique in the medical field as it enables physicians to more precisely reach the target tissue. In this paper we describe our interest in needle steering for lung biopsy and the significance of this technique. There has been much interest in modeling needle steering in recent years and this paper builds upon that work. We describe our Matlab implementation and present simulation results. We also show our experimental results based on a robotic needle driver and X-Ray imaging in the interventional suite. The experimental results showed good agreement with the simulation results.

We are witnessing a dramatic increase in the use of complex technological systems for better management and exploitation of abundant data from different sources. The Imaging Science and Information Systems (ISIS) Center, a medical research center at Georgetown University is an agile organization that is subject to fast changing requirements and new application deployments. ISIS requires an agile, secure IT infrastructure based on network management best practices that enables rapid business implementations, accommodates innovative deployments and applications, and supports its business plan. Adapting emerging technologies can facilitate communication, productivity and access but, also entails higher security risks, more assets to manage and increased requirements for compliance with established standards and security rules. In this paper we will describe the approaches we took to build and secure an IT infrastructure at the ISIS Center that enables researchers, collaborators, vendors and contractors to work in an environment that hosts systems for different purposes with no compromise to security and data confidentiality.

We report the development of a robot for radiological percutaneous interventions using uniplanar fluoroscopy, biplanar fluoroscopy, or computed tomography (CT) for needle biopsy, radio frequency ablation, cryotherapy, and other needle procedures. AcuBot is a compact six-degree-of-freedom robot for manipulating a needle or other slender surgical instrument in the confined space of the imager without inducing image artifacts. Its distinctive characteristic is its decoupled motion capability correlated to the positioning, orientation, and instrument insertion steps of the percutaneous intervention. This approach allows each step of the intervention to be performed using a separate mechanism of the robot. One major advantage of this kinematic approach is patient safety. The first feasibility experiment performed with the robot, a cadaver study of perispinal blocks under biplanar fluoroscopy, is presented. The main expected application of this system is to CT-based procedures. AcuBot has received Food and Drug Administration clearance (IDE G010331/S1), and a clinical trial of using the robot for perispinal nerve and facet blocks is presently underway at Georgetown University, Washington, DC.

This paper reports on technology developments aimed at improving the state of the art for image-guided minimally invasive spine procedures. Back pain is a major health problem with serious economic consequences. Minimally invasive procedures to treat back pain are rapidly growing in popularity due to improvements in technique and the substantially reduced trauma to the patient versus open spinal surgery. Image guidance is an enabling technology for minimally invasive procedures, but technical problems remain that may limit the wider applicability of these techniques. The paper begins with a discussion of low back pain and the potential shortcomings of open back surgery. The advantages of minimally invasive procedures are enumerated, followed by a list of technical problems that must be overcome to enable the more widespread dissemination of these techniques. The technical problems include improved intraoperative imaging, fusion of images from multiple modalities, the visualization of oblique paths, percutaneous spine tracking, mechanical instrument guidance, and software architectures for technology integration. Technical developments to address some of these problems are discussed next. The discussion includes intraoperative computerized tomography (CT) imaging, magnetic resonance imaging (MRI)/CT image registration, three-dimensional (3-D) visualization, optical localization, and robotics for percutaneous instrument placement. Finally, the paper concludes by presenting several representative clinical applications: biopsy, vertebroplasty, nerve and facet blocks, and shunt placement. The program presented here is a first step to developing the physician-assist systems of the future, which will incorporate visualization, tracking, and robotics to enable the precision placement and manipulation of instruments with minimal trauma to the patient.

Gives an overview of our program plan in robotically assisted spine procedures and details an initial cadaver study. Low back pain is a common medical problem, and minimally invasive procedures such as nerve blocks are rapidly growing in popularity as a potential method of treatment. To assist the physician in needle placement during these procedures, we have investigated using a needle-driver robot developed at the Johns Hopkins Urology Robotics Laboratory. Robotic systems such as these have great potential as physician-assisting devices for improving the precision of needle placement and enabling the development of the next generation of precision guidance systems for interventional techniques.

The United States military has been an effective proponent of digital imaging and teleradiology for the past 15 years [1]. A digital imaging network that eliminates the use of x-ray film makes military medicine requirements simpler. X-ray film requirements include storage of new, unexposed films, storage and use of chemicals and water for processing, and disposal of chemicals. In some deployed situations, the chemical discharge needs to be collected and shipped out of the area. Therefore, the ability to implement electronic imaging and eliminate or greatly reduce the dependence on film, chemicals, and water are intrinsically important to military medicine. In December 1995, the United States government began deployment of 20,000 United States troops to Bosnia-Herzegovina as part of NATO's peacekeeping implementation force (IFOR) operation. A full complement of military medical support facilities was established in Bosnia. An army base in Hungary was the location from which the deployment was staged. The project to deploy telemedicine and teleradiology capabilities to the medical treatment facilities (MTF) in Bosnia and Hungary became known as PrimeTime III [2]. This paper deals with the deployable teleradiology (DEPRAD) system that was installed by the Imaging Science and Information Systems (ISIS) Center, Department of Radiology, Georgetown Medical Center, Washington, DC, at a number of facilities to implement filmless radiology and teleradiology services in support of PrimeTime III.

The derivation of the Jacobian matrix for a novel six degree of freedom parallel manipulator is presented in this paper. A force decomposition approach is used to compute the Jacobian. The parallel manipulator consists of a base plate, a top plate, and three connecting legs with all revolute joints. The key to the Jacobian formulation presented here is the derivation of a mapping from the applied end-effector force and torque to two force components at each spherical joint. Each leg of the manipulator can then be treated separately and the actuated joint torques can easily be computed from these spherical joint forces.>

The authors examine the computed torque control problem for a robot arm with flexible, geared, joint drive systems which are typical in many industrial robots. The standard computed torque algorithm is not directly applicable to this class of manipulators due to the dynamics introduced by the joint drive systems. The proposed approach overcomes this problem by combining a novel computed torque algorithm with simple torque controllers at each joint of the robot. The control scheme is applied to a seven degree-of-freedom industrial manipulator, and the system performance in standard tasks is evaluated using both dynamic simulation and actual experiments. The results show that the proposed controller leads to improved tracking performance over a conventional PD (proportional plus derivative) controller.>

A prototype parallel link manipulator is described. The kinematics of a general parallel link manipulator are discussed. The hardware and software of the prototype manipulator are presented. Some symbolic kinematic results for the prototype are given, along with a discussion of how these results might be used. The workspace of the prototype is analyzed, both by simulation and experimentation. The singularity problem for parallel manipulators is discussed, and a singular configuration for the prototype is shown.>

The incorporation of multiple criteria in the operation of redundant manipulators is considered. The criteria investigated in the literature are categorized, and an alternative set of criteria categories is proposed. Criteria normalization and prioritization are discussed. Simulation results for a spatial seven-degree-of-freedom manipulator incorporating four criteria along a self-motion trajectory are given. An animation package for visualizing the simulation results and examining the tradeoffs between criteria is described. The study is intended as a first step toward a decision-making framework that will be used to investigate how a comprehensive set of criteria might be combined to enhance the operation of redundant manipulators.>