Surgery is primarily taught through mentoring, where an expert mentor supervises a mentee performing surgery, taking over when necessary. Telementoring systems aim to provide mentees with access to remote mentors, but the physical distance between mentors and mentees poses unique challenges to surgical training. We investigate the underlying needs leading to takeovers in onsite mentoring and assess mentors’ ability to fulfill address these needs remotely using existing telestration tools, namely pointers and drawings on shared views. Through interviews and workshops with expert surgeons, we find that (1) mentors take over to convey gestures related to instrument placement, tissue displacement, force, and movement, (2) mentors gather information about location of tissue, equipment, and instruments, as well as gesture constraints, and (3) surgeons judge telestration insufficient for these needs. Based on this gap between onsite mentoring practices and telementoring tools, we discuss novel tools to address these needs and their evaluation.

All along their careers, surgeons learn through mentoring, operating on patients side-by-side with expert surgeons. However, access to experts is more and more challenging as they are often located far from learners. Telementoring can increase the reach of mentors by supporting mentoring at a distance, but despite their potential, current systems remain rarely institutionalized. We investigate current surgical mentoring practices, to inform future design of telementoring systems in providing support for learning. In a field study, we observe 11 surgeries and conduct follow-up self-confrontation interviews with mentor-mentee pairs. Through Thematic Analysis, we find that the domain of surgery results in mentee's needs for reassurance regarding safety, being spared from workload and risk, both essential to support the acquisition of skills. Second, that mentors understand the situation primarily through the surgical site view but also through inspecting the mentee's posture and the patient's body. Thirdly, that mentors take physical distance as their presence limits mentees' autonomy development. We discuss the limits of current surgical telementoring systems and rethink their role as companions of onsite mentoring.

Cosserat rod theory proved efficient modeling performances in robotics, especially in the context of continuum robots, in the past decade. The implementation of such theory is far from being unique and straightforward. We consider the illustrative example of multisegment, general routing tendon actuated continuum robots in their nominal static operating regime. This article details two main approaches based on Cosserat rod modeling, namely, the Newtonian and Lagrangian approaches. We provide a walk-through guide regarding theoretical derivations and numerical implementation of both approaches, together with a proof of equivalence. This comparative study is supplemented with novel contributions and extensions of each approach and in-depth discussion of their performances and applicability, as well as highlighting their special features.

Interventional robotic systems have been deployed with all existing imaging modalities in an expansive portfolio of therapies and surgeries. Over the years, literature reviews have painted a comprehensive portrait of the translation of the underlying technology from research to practice. While many of these robots performed promisingly in preclinical settings, only a handful of them managed to evolve further, break through the commercialization boundary, and even fewer reached a wide-scale adoption. Despite the undeniable success of service robotics in general and particularly in some sophisticated medical applications, image-guided robotics’ impact remained modest compared to other surgical areas, especially laparoscopic minimally invasive surgery. This article aims to embrace the state of the art on the one hand, provide a comprehensive narrative of the situation described, support future system developers, and facilitate the translation from scientific research to applied clinical technology development.

Virtual and Augmented Reality Environments have long been seen as having strong potential for educational applications. However, research showing actual evidences of their benefits is sparse. Indeed, some recent studies point to unnoticeable benefits, or even a detrimental effect due to an increase of cognitive demand for the students when using these environments. In this work, we question if a clear benefit of AR and VR can be robustly measured for a specific education-related task: learning a 3D object.

Handheld Perspective-Corrected Displays (HPCDs) are physical objects that have a notable volume and that display a virtual 3D scene on their entire surface. Being handheld, they create the illusion of holding the scene in a physical container (the display). This has strong benefits for the intuitiveness of 3D interaction: manipulating objects of the virtual scene amounts to physical manipulations of the display. HPCDs have been limited so far to technical demonstrators and experimental tools to assess their merits. However, they show great potential as interactive systems for actual 3D applications. This requires that novel interactions be created to go beyond object manipulation and to offer general-purpose services such as menu command selection and continuous parameter control. Working with a two-handed spherical HPCD, we report on the design and informal evaluations of various interaction techniques for distant object selection, scene scaling, menu interaction and continuous parameter control. In particular, our design leverages the efficient two-handed control of the rotations of the display. We demonstrate how some of these techniques can be assemble in a self-contained anatomy learning application. Novice participants used the application in a qualitative user experiment. Most participants used the application effortlessly without any training or explanations.

The feeling of presence of virtual entities is an important objective in virtual reality, teleconferencing, augmented reality, exposure therapy and video games. Presence creates emotional involvement and supports intuitive and efficient interactions. As a feeling, presence is mostly measured via subjective questionnaire, but its validity is disputed. We introduce a new method to measure the contribution of several technical parameters toward presence. Its robustness stems from asking participant to rank contrasts rather than asking absolute values, and from the statistical analysis of repeated answers. We implemented this method in a user study where virtual entities were created with a handheld perspective corrected display. We evaluated the impact on two virtual entities' presence of four important parameters of digital visual stimuli: resolution, latency, frame rate and jitter. Results suggest that jitter and frame rate are critical for presence but not latency, and resolution depends on the explored entity.

In the context of developing a pedagogical tool for teaching anatomy, the need for a comparative study between two RGB-D cameras has emerged. This paper addresses the assessment of the accuracy and precision of two RGB-D sensors (Carmine Primesense and Persee Orbbec) through two different experiments. The evaluation not only provides comparative results on the sensors performances but also aims at determining in which conditions they are the most efficient. The first experiment evaluates the variability of the output depth map data. The second experiment focuses on analyzing the influence of the distance in the positioning accuracy of an object submitted to controlled displacements. The results are summarized in a set of error heat maps and a table; they provide clues for using one sensor rather than another by describing their robustness both in a static scene and in a motion capture scenario.

This paper introduces a new robust 3D ultrasound needle detection approach integrated in a 3D needle steering system associated to a real-time path planning. The robustness of an existing algorithm is improved by limiting the needle detection to a curvilinear region of interest (ROI) using a novel mechanical-based prediction model. This linear model is also used in a Kalman filter to reduce detection noise and reject false detections. These two improvements drastically increase quality of our feedback. Finally, the 3D needle steering system is able to reach a target in phantoms with a maximal error of 0.8 mm without obstacle and 1.6 mm with obstacle.

Ultrasound (US) probes have been used as guiding tools for Computer Assisted Orthopedic Surgeries (CAOS) [1]. Because of the US data uncertainty, the process of recognition - the localization of regions of interest in the image-requires a registration to a more precise, but invasive, imaging modality such as Computed Tomography (CT). A millimetric precision and a real-time processing are intraoperative requirements. Iterative Closest Point (ICP) [2] is a simple and non symmetric rigid registration algorithm that is sensitive to the initial position of the point sets. The aim of this study is to show the contribution of initializing ICP in rigid US-CT registration and to illustrate it on data of a proximal femur. First, an iterative initialization of the model (CT) to the partial view (US) is performed using ICP with annealed filtering. The first obtained local minimum is then used to initialize a refinement step that maps the partial view to the model. One femur phantom was imaged both in a water bath using a calibrated 3D ultrasound probe and by CT. For each of the ten US acquisitions (five in the Anterior neck A, and five in the Posterior neck P), the CT scan is brought by means of fiducials pair-point matching. The initialization step improves ICP successful registrations from (A:25%, P:21%) to (A:76%, to P:52%) and the registration takes about 3s in average whilst ICP takes about 1s.

This paper presents a mirror-like augmented reality (AR) system to display the internal anatomy of a user. Using a single Microsoft V2.0 Kinect, we animate in real-time a user-specific internal anatomy according to the user's motion and we superimpose it onto the user's color map, as shown in Fig.1.e. The user can visualize his anatomy moving as if he was able to look inside his own body in real-time.

The recent availability of navigation systems for mapping and targeting of transrectal ultrasound (TRUSS) guided prostate biopsies revealed new opportunities in training the clinician. This paper describes a simulator for TRUSS guided prostate biopsy that offers similar information, enhanced by a complete learning environment. Various exercises have been developed in accordance with a didactical study identifying the training needs. A dedicated clinical case database fed by a prostate navigation system provides a large patient prostate image database that covers the main situations encountered during clinical practice. A haptic device is used to enable complete biopsy procedures or practice specific tasks. This paper also presents work in progress of the evaluation of such a simulator.

This paper describes a new 3-D ultrasound robotic prostate brachytherapy system. It uses a stationary 3-D ultrasound probe rigidly fixed to a robotic needle insertion mechanism. The novelty of the system is its ability to track prostate motion intraoperatively to allow the dose planning and needle trajectories or depths to be adapted to take into account these motions. Prostate tracking is done using a fast 3-D ultrasound registration algorithm previously validated for biopsy guidance. The 7-degree-of-freedom (7-DOF) robot and ultrasound probe are calibrated together with an accuracy of 0.9 mm, allowing the needles to be precisely inserted to the seed targets chosen in the reference ultrasound image. Experiments were conducted on mobile, deformable synthetic prostate phantoms using a prototype laboratory system. Results showed that, with prostate motions of up to 7 mm, the system was able to reach the chosen targets with less than 2-mm accuracy in the needle insertion direction. This measured accuracy included extrinsic measurement errors of up to 1.1 mm. A preliminary cadaver feasibility study was also described in preparation for more realistic experimentation of the system.

Until recently, computer-aided medical interventions (CAMI) and medical robotics have focused on rigid and nondeformable anatomical structures. Nowadays, special attention is paid to soft tissues, raising complex issues due to their mobility and deformation. Mini-invasive digestive surgery was probably one of the first fields where soft tissues were handled through the development of simulators, tracking of anatomical structures and specific assistance robots. However, other clinical domains, for instance urology, are concerned. Indeed, laparoscopic surgery, new tumour destruction techniques (e.g., HIFU, radiofrequency, or cryoablation), increasingly early detection of cancer, and use of interventional and diagnostic imaging modalities, recently opened new challenges to the urologist and scientists involved in CAMI. This resulted in the last five years in a very significant increase of research and developments of computer-aided urology systems. In this paper, we propose a description of the main problems related to computer-aided diagnostic and therapy of soft tissues and give a survey of the different types of assistance offered to the urologist: robotization, image fusion, surgical navigation. Both research projects and operational industrial systems are discussed

LER is a compact surgical assistant robot for positioning of an endoscope and camera during minimally invasive surgery. In contrast to typical endoscope manipulators, LER is particularly compact and lightweight at 625 g and 110 mm in diameter, so that it is simple to set up and use, occupies no floor space, and does not limit access to the patient in any way. Our current prototype is fully sterilizeable by autoclave and is ready for clinical trials. It features a full motion range of 360/spl deg/ in rotation and inclination to 10/spl deg/ from the horizontal plane and is backdriveable for manual positioning. Actuation forces are limited for safety. LER may be held in place on the abdomen by adhesive strips or sutures, or attached to the sides of the table with elastic straps or clamps. We have implemented a variety of different user command interfaces for LER, including a miniature keypad, automatic optical instrument motion tracking, and voice command recognition. Experimental trajectory following results and performance parameters are given.

This paper presents a slave robot carrying an ultrasound probe for remote echographic examination. This robot is integrated in a master-slave system called robotic tele-echography (TER). The system allows an expert operator to perform a remote diagnosis from echographic data he acquires on a patient located in a distant place. The originality of this robot lies in its architecture: the cable-driven robot is lightweight and semirigid, and it is positioned on the patient body. In this paper, we describe the clinical application, the system architecture, the second implementation of the robot, and experiments performed with this prototype.

We have developed a cable-driven manipulator using pneumatic artificial muscle actuators to control the orientation and insertion depth of an endoscope during abdominal surgery. This manipulator enables a single surgeon to manipulate surgical instruments with both hands while the endoscope position is controlled to view an area of interest inside the abdomen. The surgeon may then control the endoscope by alternate methods such as voice commands, pedals, a joystick, or head movements. The advantages of our newly developed endoscope manipulator over those of commercially available robotic laparoscopic surgical systems are its low cost, simplicity, ease of setup and use, compliance, nonintrusiveness, and very small size and light weight, with the disadvantage of somewhat reduced absolute positioning accuracy. Multiple instruments may be manipulated simultaneously by the manipulators set close together. Experimental performance results of open-loop and feedback control methods are presented.

Computer assisted surgery (CAS) attempts primarily to optimize the performance of medical tasks. This consists in planning optimal strategies and accurately transferring and executing these plans in intra-operative conditions. One clinical purpose is to minimize the intervention invasiveness by increasing the selectivity of anatomical target localization and reducing the aggressiveness of the target access through minimal incisions or natural openings. This generally results for the surgeon in a lack of perception (vision, tactile feeling) and dexterity. It may also raise accessibility problems and may require a higher positional and force accuracy. Therefore, an interface device of some sort is needed in order to connect the "information world" of images, plans, and computers, to the physical world of surgeons, patients, and tools. Such a device is called a "guiding system". We discuss the PADyC system whose purpose is to limit the possible actions of the surgeon to what is strictly authorized in the planning. PADyC relies on man-machine cooperation. We present its principles and we discuss its application to cardiac punctures.

Whilst computers are more and more used in the medical and surgical domains, medical education is still very conservative. However, new information technology could bring a lot to this area. In particular, simulators are of paramount interest: they would allow one both to experiment with surgical or imaging techniques for a wide range of pathologies without the need for patients and to quantitatively evaluate the trainees' performances. Ultrasound imaging is a typical area where virtual reality may change educational uses. This paper describes an ultrasound imaging simulator dedicated to the training of physicians for the detection of deep venous thromboses of the lower limbs. Like this one, a lot of other pathologies of soft tissue are diagnosed using ultrasound imaging. Because this examination is difficult and operator-dependent, developing a simulator is very useful to give common databases of pathological samples on which physicians can both experiment with image acquisition and evaluate their understanding of clinical cases. Real-time simulation is mandatory. An ultrasound imaging simulator has been developed and is being tested. This paper presents the hardware and software components of the simulator and describes the status of this project. Copyright © 2000 John Wiley & Sons, Ltd.

Describes a system based on the combined use of medical imaging and robot positioning used in stereotactic neurosurgery. The types of interventions performed with the help of this system require a high precision (less than 1 millimeter) of positioning with respect to the patients brain with no direct visibility of the target; they are executed through a hole in the skull of 2.3 millimeters. A typical example is the placement of a stimulating electrode in a particular nucleus of the thalamus for patients suffering from Parkinson's disease. These interventions rely on the intensive use of medical imaging to define the operative strategy. The target is defined with respect to the images. A six-axis robot whose end-effector is a linear guide automatically reaches a configuration generated such that if the surgical instrument (needle, electrode, etc.) is introduced in the guide, the surgeon reaches the target using a linear displacement of the instrument. This system is operational and is in routine clinical use.>

The authors describe ACT, a system designed to plan and monitor robot motions using a geometric representation of the environment. The goal in developing ACT was to design a system which could be used in industrial applications as well as a software basis to develop new robot programming techniques in the academic environment. ACT is intended to provide the user with advanced programming tools such as inline path planning and graphic monitoring of robot tasks. The project included developments not directly related to robotics but which were essential to enhance these new programming techniques. For example, a special user interface was developed to specify the robot tasks as well as to interface with existing CAD modelers. The authors present the technical choices made in developing the system and the most technically innovative parts of the system.>

Observations of human grasping show two phases. During the reaching phase of grasping, the hand preshapes in order to prepare for the second phases which is shape-matching with the object. Planning of grasping with dextrous robot hands cannot be summarized to these two phases. It is necessary to split the grasping process into several phases (frequently overlapped), and to consider problems such as object recognition, planning accessibility, task planning, initial touch and grab phase, and stable grasp phase. These are consciously or unconsciously generated by a human being. A major issue is to integrate a part of these components in preshaping, and in particular in automatic planning accessibility and preshaping of objects to be grasped. The system is based on 2D analysis of slices extracted from an octree representation of objects.>

This paper deals with the problem of planning safe trajectories for a manipulator robot moving in an environment including both static obstacles and objects (other robots, conveyors) whose positions may change during the interval of time separating two planning steps. In order to solve this problem for a general manipulator having six degrees of freedom, the authors have developed and implemented a method combining two different path planners: a local planner whose purpose is to take the manipulator away from the constrained initial and final configurations, and a global planner which is in charge of generating a safe trajectory for the first three degrees of freedom of the arm. Both the local and the global planners are based on a fast iterative algorithm for computing the distance between pairs of objects. These planners have been implemented within a robot programming system (called ACT) including a solid modeler dedicated to robotics applications and a connection to a robot controller.>

The problem of making sensing and acting techniques cooperate in order to achieve a given manipulation task in a partially structured environment is treated in the context of automatic grasping by guiding the decisional process using a combination of partial geometric models and a vision data. The geometric models represent the known information concerning the robot workspace and the object to be grasped. The vision-based information is collected at execution time using a 2D camera and a 3D vision sensor, both located on the robot end effector. This means that robot motions and sensing operations have to be combined for the purpose of both acquiring the missing information and guiding the grasping movements. This is achieved by applying three processing phases respectively aimed at selecting a viewpoint avoiding occlusions, modeling the local environment of the object to be grasped, and determining the grasping parameters.>

Despite much research in the field, the use of automatic robot programming systems is still a long term issue; it is the reason why we think that, right now, we have to use the methodology supplied by this research to develop object level on-line robot programming tools. Moreover, we think that some accessibility problems which automatic robot programming addresses might be solved a better way in a sensor based approach. In this paper, we focus on the problem of accessibility analysis for automatic grasping. The problem is to choose a goal configuration and a safe path for a 3D object moving bi-dimensionally among 3D obstacles. We propose an approach relying on a bidimensional geometric reasoning; our approach consists in discretizing the space swept by the gripper using a set of parallel planes. Then, an accessibility analysis using the Configuration Space approach is performed in each of these planes and local constraints are transformed in constraints on the motion of the whole gripper. In this paper, we describe this approach and the work in progress for using visual data in building a model of the empty space.