Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

Minimally Invasive Surgery (MIS) is one of the most successful applications of surgical robots. Although the introduction of robotic technology has brought a number of benefits, further advancements in MIS are limited by the size and bending radius of instruments. In this paper, we present a compliant four degree-of-freedom manipulator that consists of elastic elements with partly thinner structures. The proposed mechanism allows the elastic element to deform locally, thus minimizing its bending radius while the low number of mechanical parts greatly contributes to its compactness. This paper describes the design strategy, optimization method using FEA, prototype implementation, and evaluations. The evaluations reveal high accuracy and repeat accuracy, which are key elements for robotic instruments in MIS. Further, the prototype is able to exert sufficient force and it is possible to perform a simulated needle insertion task using the manipulator, demonstrating the feasibility of the proposed mechanism.

The Rubber Hand Illusion is a well known multisensory illusion in which the participant feels their own unseen hand become a rubber hand that is placed at the position where their hand is supposedly placed, triggered by synchronized haptic stimuli on their hand and the rubber hand. Our previous study showed that a robotized 1-degree-of-freedom rubber hand in the form of a master-slave system significantly enhanced the illusion compared with the classic rubber hand illusion. The illusion gives the sense that the part of body becomes part of the robot - it is thus assumed that this multisensory illusion is a potentially effective way to increase intuitiveness, and therefore increase the operability of a multi-degrees-of-freedom master-slave system. To further investigate the concept, we built a multisensory illusion inducible microsurgical robotic system based on the findings of the previous study and conducted a series of psychophysical experiments. The experimental results reveal that the multisensory illusion and operability is significantly correlated.

Minimally invasive surgery (MIS) is commonly used in pediatric operations. This method greatly benefits patients because of the reduced surgical trauma. To perform such surgery smoothly, doctors must be highly skilled. To reduce operating difficulties, a great deal of research on surgical systems have been carried out. However, in some cases, smaller workspaces limit the application of MIS. For example, the workspace of infant congenital esophageal atresia (ICEA) surgery is only around 30×30×30 mm. Until now, most ICEA surgeries have been manually performed with traditional instruments. This paper presents a smart surgical robot (SSR) for ICEA surgery. The robot is composed of two slave arms, each consisting of a positioning manipulator and a surgical tool manipulator. The positioning manipulator uses a selective compliance assembly robot arm (SCARA) and a screw-pair mechanism to achieve translational movement in 3D space, and the surgical tool manipulator uses a “double screw drive + universal joint” structure to allow an omni directional bending motion. During surgery, the surgeon first creates the workspace manually to explore the target esophagus. The SSR system is then applied to perform operation. The configuration of the SSR means it can perform tissue manipulation under endoscopic view in a small workspace. Experimental results show that the endoscopic view permits the SSR system to be operated intuitively and accurately in the target workspace.

We herein describe a needle insertion method involving tissue preloading for accurate breast tumor treatment. A mechanical preloading probe locates a tumor lesion from ultrasound imaging information and reduces lesion displacement during needle insertion by pressing the breast tissue. We validated the insertion accuracy of this method by numerical simulation and experiments both in vitro and in vivo. For further accuracy enhancement, we evaluated the geometry effect of the preloading probe on needle insertion accuracy by experiments in vitro. We compared the insertion accuracy between insertion with preloading using different probe diameters and normal needle insertion. In addition, we compared insertion accuracy at different tumor depths. The data indicated a tendency for adaptation of larger preloading probe diameters with deeper tumors. This suggests the potential for our method to enhance placement accuracy by real-time geometry regulation.

Recently, increased attention has been focused on single port endoscopic surgery (SPS). We have developed a robotic system for SPS with two surgical manipulators: an endoscopic manipulator and a positioning manipulator that moves the endoscope. The robot can manipulate both the position and orientation of the endoscope to achieve the desirable endoscopic field of view. Two methods can be used to operate the endoscopic view: “control corresponding to position” mode and “control corresponding to velocity” mode. Although both are widely used for moving the visual field, the operability of each method has not been examined quantitatively. Thus, we compare the operability of the two methods for adjusting the endoscopic view, and present the results and suitable applications of each method. The results of the quantitative evaluation experiments show that the “control corresponding to position” mode is suitable for short-distance or precise adjustment of the endoscope, whereas the “control corresponding to velocity” mode is better suited to long-distance movement.

The use of minimally invasive procedures for breast tumor diagnosis and treatment, such as needle biopsy and radiofrequency ablation (RFA), is steadily increasing. Accurate needle insertion requires solving the problems of tissue deformation and target displacement. In this study, we developed a robotic needle insertion method to improve the precision of diagnostic biopsy and RFA treatment. The mechanical probe was designed to reduce tissue displacement by pressing the breast tissue before needle insertion: a technique that is known as preloading. We focused on the needle insertion phase and evaluated the insertion accuracy achieved. Using a numerical simulation model and an actual hog breast, we compared tissue preloaded needle insertion with normal needle insertion. The data obtained with both test systems showed that targeting errors were greatly reduced using preloading-based needle insertion, as compared with normal needle insertion. The procedure is expected to offer a safe and effective alternative to the traditional methods of needle insertion for breast tissue biopsy or RFA. Our study also revealed the relationship between insertion accuracy and preloading probe force.

We describe a palpation-based needle insertion method for diagnostic biopsy and radiofrequency ablation treatment of a breast tumor. The mechanical palpation probe locates cancerous tissue from ultrasound imaging information and reduces tissue displacement during needle insertion by pressing the breast tissue. We examined the insertion accuracy of this method and compared palpation-based needle insertion with normal needle insertion both in vitro and in vivo. Palpation-based needle insertion had a smaller error, suggesting that this procedure is a safe, effective alternative to traditional methods of breast needle insertion.

Performing laparoscopic surgery requires several skills which have never been required for conventional open surgery, surgeons experience difficulties in learning and mastering these techniques. Various training methods and metrics have been developed in order to assess and improve surgeon's operative abilities. While these training metrics are currently widely being used, skill evaluation methods are still far from being objective in the regular laparoscopic skill education. This paper proposes a methodology of defining a model to objectively evaluate surgical performance and skill expertise in the routine laparoscopic training course. Our approach is based on the processing of kinematic data describing the movements of surgeon's upper limbs. An ultra-miniaturized wearable motion capture system (Waseda Bioinstrumentation system WB-3), therefore, has been developed to measure and analyze these movements. The skill evaluation model was trained by using the subjects' motion features acquired from WB-3 system and further validated to classify the expertise levels of the subjects with different laparoscopic experience. An experiment for training fundamental laparoscopic psychomotor skill was elaborated by using laparoscopic box trainer. Preliminary results show that, the proposed methodology can be efficiently used both for quantitative assessment of surgical ability, and for the discrimination between expert surgeons and novices.

Force feedback systems have been developed to improve the user-friendliness of robotic surgery systems. In teleoperative surgery, one of the major problems is the delay of the transmission of the information from the operation site to the surgery site. The transmission delays of the control signal and force information are less than the that of the image signal. In many cases, the visual information is synchronized with the force information in order to avoid a time gap between them. In this paper, we propose that force information be presented earlier than the visual information. This approach reduces the damage to the organs in surgical operations even though there is a time gap between the delay of force information and visual information. In addition, we propose a method to present predicted force information to avoid damage to the organs due to time delays in teleoperative surgery. The liver can be treated with a contact force of less than 0.8 N and a grasping force of less than 1.1 N by applying the proposed method. The method can realize a teleoperative surgery without damaging the patient's organs.

We describe here a palpation-based needle insertion method for diagnostic biopsy and treatment of breast cancer. The mechanical palpation probe locates cancerous tissue from force information and reduces tissue displacement during needle insertion. We compared palpation-based needle insertion to normal needle insertion by numerical simulation of a breast tissue model and by experiments in vitro. The data showed palpation-based needle insertion had a smaller error in both tests. Our findings suggest the procedure is a safe, effective alternative to traditional methods of breast tissue biopsy.

This paper describes a ultrasound (3-D US) system that aims to achieve augmented reality (AR) visualization during laparoscopic surgery, especially for the liver. To acquire 3-D US data of the liver, the tip of a laparoscopic ultrasound probe is tracked inside the abdominal cavity using a magnetic tracker. The accuracy of magnetic trackers, however, is greatly affected by magnetic field distortion that results from the close proximity of metal objects and electronic equipment, which is usually unavoidable in the operating room. In this paper, we describe a calibration method for intraoperative magnetic distortion that can be applied to laparoscopic 3-D US data acquisition; we evaluate the accuracy and feasibility of the method by in vitro and in vivo experiments. Although calibration data can be acquired freehand using a magneto-optic hybrid tracker, there are two problems associated with this method--error caused by the time delay between measurements of the optical and magnetic trackers, and instability of the calibration accuracy that results from the uniformity and density of calibration data. A temporal calibration procedure is developed to estimate the time delay, which is then integrated into the calibration, and a distortion model is formulated by zeroth-degree to fourth-degree polynomial fitting to the calibration data. In the in vivo experiment using a pig, the positional error caused by magnetic distortion was reduced from 44.1 to 2.9 mm. The standard deviation of corrected target positions was less than 1.0 mm. Freehand acquisition of calibration data was performed smoothly using a magneto-optic hybrid sampling tool through a trocar under guidance by realtime 3-D monitoring of the tool trajectory; data acquisition time was less than 2 min. The present study suggests that our proposed method could correct for magnetic field distortion inside the patient's abdomen during a laparoscopic procedure within a clinically permissible period of time, as well as enabling an accurate 3-D US reconstruction to be obtained that can be superimposed onto live endoscopic images.

Remote surgery is one of the most desired applications in the context of recent advanced medical technologies. For a future expansion of remote surgery, it is important to use conventional network infrastructures such as Internet. However, using such conventional network infrastructures, we are confronting time-delay problems of data transmission. In this paper, a remote surgery experiment between Japan and Thailand using a research and development Internet is presented. In the experiment, the image and audio information was transmitted by a newly developed low latency CODEC system to shorten the time-delay. By introducing the low latency CODEC system, the time-delay was shortened compared with the past remote surgery experiments despite the longer distance. We also conducted several network measurements such as a comparison between TCP/IP and UDP/IP about the control signal transmission.

Several robotic surgical systems have been developed for MIS (minimally invasive surgery) including commercialized products such as da Vinci and ZEUS. We have developed a minimally invasive surgical system, which have carried out remote surgery experiments for five times at the writing time. In this paper, a remote surgery experiment, which was conducted between Japan and Korea by using the developed minimally invasive surgical system is described. Research & Development (R & D) Internet testbet, APII (Asia-Pacific Information Infrastructure), which consists of an optical submarine cable network KJCN (Korea-Japan Cable Network), was used. In the experiment, a laparoscopic cholecystectomy was successfully carried out on a pig. The network time-delays of control signal and images were 6.5 msec and 435.5 msec respectively. A comparison of remote surgery experiments using ISDN and the Internet was studied

Several robotic surgical systems have been developed for minimally-invasive surgery including commercialized products such as da Vinci and ZEUS. We have developed a minimally-invasive surgical system, which had performed remote surgery experiments for four times until now (performed on pigs over 150 km in Japan). In this paper, two approaches that extend the system from the perspective of extensions of surgeon's dexterity are described. The first approach is a development of complex robotic forceps, which have multiple d.o.f. on their tip. A complex rigid link mechanism has been applied to the robotic forceps to realize 3 d.o.f. motions in a patient's abdominal cavity. With the rigid link mechanism, surgeons have no longer need to be concerned a cable cutting problem, which often can be happened on conventional cable driven systems. The second approach is a force feedback capability. The developed rigid link mechanism is preferable for an implementation of a force feedback capability. The force feedback capability is important not only for extending surgeon's dexterity but also for preventing unexpected damage on internal organs. An evaluation experiment of the force feedback capability on a simulated surgical procedure was conducted. An augmented force presentation method was developed that displays contact information between forceps and soft tissues. In an evaluation experiment, the load applied to soft organ tissues was reduced by approximately 30% compared with a conventional force feedback method.

This paper describes a tele-endoscopic surgical system with force feedback capability. The system consists of multi-media cockpit, surgical site and communication link. The multimedia cockpit is equipped with force feedback type master manipulators, visual and auditory information presentation apparatuses, and foot switches. A slave manipulator with 3 arms is located at the surgical site; the left and right arms hold forceps or a radio knife and the center arm holds an endoscope. The slave manipulator has a special safety mechanism. Vital signs can be also transmitted to the cockpit. In the experiment, a cholecystectomy was successfully performed on a pig at the distance of 150 km.