In this paper, we propose a probabilistic motion model for tendon actuated continuum robots that experience actuation transmission non-linearities due to cable slack and cable-sheath friction. The model is based on a Lie group formulation of the robot’s end-effector pose that incorporates a new simple backlash model. Bayesian parameter estimation is then employed to learn a probability distribution over the model’s parameters. This allows the uncertainty over the parameters to be propagated in the prediction of the end-effector’s trajectory. The model’s predictive capabilities are compared against the static Cosserat-rod-based model and the Kirchhoff model in simulation and are validated with experiments on a robotized medical endoscope.

Optical Coherence Tomography (OCT) is increasingly used in endoluminal procedures since it provides high-speed and high resolution imaging. Distortion and instability of images obtained with a proximal scanning endoscopic OCT system are significant due to the motor rotation irregularity, the friction between the rotating probe and outer sheath and synchronization issues. On-line compensation of artefacts is essential to ensure image quality suitable for real-time assistance during diagnosis or minimally invasive treatment. In this paper, we propose a new online correction method to tackle both B-scan distortion, video stream shaking and drift problem of endoscopic OCT linked to A-line level image shifting. The proposed computational approach for OCT scanning video correction integrates a Convolutional Neural Network (CNN) to improve the estimation of azimuthal shifting of each A-line. To suppress the accumulative error of integral estimation we also introduce another CNN branch to estimate a dynamic overall orientation angle. We train the network with semi-synthetic OCT videos by intentionally adding rotational distortion into real OCT scanning images. The results show that networks trained on this semi-synthetic data generalize to stabilize real OCT videos, and the algorithm efficacy is demonstrated on both ex vivo and in vivo data, where strong scanning artifacts are successfully corrected.

Non-linearities in cable transmissions are important limitations for the accurate control of flexible instruments used in medical endoscopic systems. Hysteresis effects greatly impact the accuracy of conventional kinematic models. This is especially critical for implementing automatic motions in flexible medical robotic systems. In this paper, we propose a method for improving open-loop accuracy of flexible instruments by implementing a Position Inverse Kinematic Model which is able to take into account hysteresis effects. In order to avoid complex physical modeling, the method relies on the off-line learning of the behavior of the instruments. Basic knowledge of the kinematic is also incorporated in the learning process in order to make it fast. The validity of the approach is demonstrated by the execution of 2D and 3D trajectories with the instruments of the STRAS medical robot. The accuracy is shown to be significantly improved with respect to other learning-based methods.

The spatial configuration of actuated flexible instruments is fundamental for control applications in robotic noscar surgery. In these operations, the instruments are inserted in the channels of a flexible guide equipped with an endoscopic camera. In this paper we propose to estimate the position of the instruments of the Anubis platform (Karl Storz) using the endoscopic images provided by the embedded camera. In this system flexible instruments have 3-DOF (translation and rotation in the channel and deflection). Application of standard approaches for 3D location with such system do not provide good accuracy because of uncertainties on several model parameters. To cope with these uncertainties, supervised learning methods has been explored. With the help of colored visual markers attached to the instruments, the proposed approach consists of an image segmentation stage followed by a position estimation stage. Firstly, the markers are segmented in the images using an AdaBoost classifier manually trained on in-vivo images. Subsequently, the resulting blobs are used as input data of an approximation function trained using ground truth information provided by a magnetic sensor. A comparison with two other model-based methods showed the potentialities of such an approach on real devices.

In this paper we present a new robotic system, called STRAS, designed for endoluminal and transluminal surgery. The system is based on the Anubis®platform - a manual flexible system designed by Karl Storz for transluminal operations. Our new robot is a modular robotic system, compatible with the medical environment, allowing an easy setup in the operating room. It provides up to 10 Degrees of Freedom (DoFs), enabling the 3D positioning of an endoscopic camera, positioning of two instruments and offering grasper opening / closing functionalities. The paper presents for the first time the mechanical, electrical and control design of STRAS. An initial characterization of the system shows that, because of complex mechanical interactions, the kinematic modeling is not sufficient for cartesian control. However, first experiments, demonstrate that the robotic system can be telemanipulated using joint control and that the robotic system enables a single user to perform complex tasks with the underlying flexible system.

Stras is a flexible robotic system based on the Anubis® platform from Karl Storz and is aimed for intralu-minal and transluminal procedures. It is composed of three cable-driven sub-systems, one endoscope and two insertable instruments. The bending instruments have three degrees of freedom and can be teleoperated by the user via two commercial master interfaces (Omega.7, Force Dimension). In this paper we investigate several ways to map the motions from the master side to the instruments, from joint per joint control to cartesian control. We describe these mappings and compare them in elementary tasks in an attempt to analyze how non-linearities affect the accuracy of control. Results show that joint control and pseudo-cartesian control provide equivalent accuracy but with different difficulties for the user.

The use of flexible cable-driven systems is common in medicine (endoscope, catheter...). Their flexiblity allows surgeons to reach internal organs through sinuous and constrained ways. Unfortunately these systems are subject to backlash due to their internal mechanism. These non linearities raise many difficulties when robotizing and controlling such systems. In this article we propose an approach to improve the cartesian control of a four ways flexible endoscopic system with strong and unknown backlash-like non linearities. The method is based on an automatic off-line hystereses learning. We show that, despite coupling between degrees of freedom, it is possible to extract information from the hystereses which allow to improve cartesian control. Experiments on a real endoscopic system show the validity and the interest of the approach.

Flexible endoscopes are used in many diagnostic exams, like gastroscopies or colonoscopies, as well as for small surgical procedures. Recently, they have also been used for endoscopic surgical procedures through natural orifices [natural orifice transluminal endoscopic surgery (NOTES)] and for single-port-access abdominal surgery (SPA). Indeed, flexible endoscopes allow access of operating areas that are not easily reachable with only one small external or internal incision. However, their manipulation is complex, especially for surgical interventions. This study proposes to motorize the flexible endoscope and to partly robotize its movements in order to help the physicians during such interventions. The paper explains how the robotized endoscope can be used to automatically track an area of interest despite breathing motion. The system uses visual servoing and repetitive control strategies and allows stabilization of the endoscopic view. All required parameters are automatically estimated. In vivo experiments show the validity of the proposed solution for the improvement of the manipulation of flexible endoscopes.

The use of flexible endoscopes in new surgical procedures such as NOTES, i.e. Natural Orifice Transluminal Endoscopic Surgery, raises many problems. Indeed, the movements of conventional flexible endoscopes are limited and surgeons can only perform basic tasks with these systems. In order to enhance endoscope possibilities and workspace, we are currently developing a robotized system. The prototype we propose is based on the combination of several flexible endoscopes. The kinematic model of the system is detailed in order to develop a method of control. The paper presents the implementation of a control strategy using an external sensor which allows to deal with non linearities induced by the cable mechanism of the endoscopes.

Estimating insertion points of surgical instruments for minimally invasive surgery is a necessary step to be able to control surgical instruments using endoscopic images. In this paper, we propose an analysis of possible methods which use image information only. Mathematical properties are detailed together with statistical properties obtained by simulations. Then a specific method is chosen to estimate the insertion point for bi-modal surgery (laparoscopy and flexible endoscopy). In vitro experiments show the accuracy of the approach and how it is possible to track the motion of the insertion point in the case of physiological motions.

Flexible endoscopes are used in many surgical procedures and diagnostic exams. They have also been used recently for new surgical procedures using natural orifices called NOTES. While these procedures are really promising for the patients, they are really awkward for the surgeons. In order to assist the surgeon, physiological motion cancellation has been successfully applied on a robotized endoscope in L. Ott, et al., (May 19-23, 2008), by using a prototype repetitive controller (PRC) and a repetitive generalized predictive controller (R-GPC). Both controllers showed to be powerful tools to cancel periodic disturbances but with poor transient response to non-periodic disturbances. Contrary to the R-GPC, the PRC is unsuitable for handling non-periodic reference changes. We propose in this paper, as a first improvement, a model-based control scheme using the PRC which allows to decouple the reference tracking from the periodic output disturbance rejection. The response to nonperiodic disturbance is also improved by this technique but a repetition appears caused by the repetitive controller. As a second improvement, a switching control scheme is proposed to avoid the repetition.

Flexible endoscopes are used in many surgical procedures and diagnostic exams, like in gastroscopy or coloscopy. They have also been used recently for new surgical procedures using natural orifices called NOTES. While these procedures are very promising for the patients, they are quite awkward for the surgeons. The flexible endoscope allows the access to operating areas which are not easily reachable, with small or no incisions; but the manipulation of the system is complex. In order to help the practicians during NOTES or classical interventions with flexible endoscopes, we propose to motorize the system so as to partially robotize the movements. This paper presents the problems in the use of the flexible endoscope and explains how the system can be used to stabilize the endoscope on an area of interest despite physiological motions and therefore to improve the manipulation of the system.

This paper deals with the visual servoing of textured surfaces inside the human abdomen with a laparoscope for the robot-assisted minimally invasive surgery (MIS). The well-known image-based visual servoing (IBVS) is one of the most common approaches used for model-based servoing. When no CAD model is available, the efficient second-order minimization (ESM) tracking developed by Malis (2002, 2004) for grey-level images is one of the powerful recent techniques which is extended here to color images so as to handle occluded parts of the region of interest (ROI). Firstly, the ROI is splitted into small areas and a histogram-based color feature comparison of image areas is presented. For each frame and for each area, a metric based on the Bhattacharyya criterion is used to select the contributing areas for the computation of the planar homography between views. Secondly, since for any MIS technique, the endoscopic lens is passing through an insertion point on the abdominal wall, a specific control strategy is developed to perform the ESM tracking with a 4-DOF surgical robot. The method presented in this paper has been validated with several video sequences. Experimental results show that the tracking method is efficient even with more than 75 % of the tracked ROI occluded. Finally, the model-free visual servoing has been performed with the AESOP surgical robot and a training box. Even if the convergence rate is a little bit slow, the desired region is always reached.

Medical robotics is a field where dedicated mechanisms have an increasing importance. The strong operating room constraints, both medical and practical, lead to heavily customized solutions. In this paper, we consider the practical problems that must be solved to build a robotic system dedicated to medical interventions under CT-scan guidance. This compact robotic assistant is placed on the patient what raises new generic robotic design problems. Practical solutions together with the evaluation of a prototype are presented in this paper

Robot-assisted surgical procedures require to generate paths for surgical instruments during a planning step and then to follow automatically the computed trajectories as close as possible. Most of the existing procedures use external localization devices to control the robot. However, these devices are not adequate for minimally invasive interventions such as laparoscopic surgery because this kind of surgery involves large lever effects which make high accuracy unreachable. In this paper, we propose a path following method based on the use of the endoscopic camera and instruments with markers. It mainly stands on an image-based visual servoing scheme and on the estimation of the 3D position of the incision point in the abdominal wall. We show that this method can lead to precise tracking despite the errors on the position of the incision point. The proposed method is finally demonstrated in vitro in an automatic suturing intervention.

The work presented in this paper addresses the problem of the stitching task in laparoscopic surgery using a circular needle and a conventional 4 DOFs needle-holder. This task is particularly difficult for the surgeons because of the kinematics constraints introduced by the trocar. So as to assist the surgeons, we propose to compute possible pathes for the needle through the tissues, which limits as much as possible tissues deformations while driving the needle towards the desired target. The article proposes a kinematic analysis and a geometric modeling of the stitching task. Based on this, some simple but useful information can be obtained to help the surgeon. The description of the task with well-chosen state variables allows to simply express the path planning problem. Conditions for the existence of acceptable pathes are given and a method to compute possible pathes is presented. Resulting pathes are shown to be satisfactory even under awkward configurations.

In this paper, we present a method to plan and control the 6 DOF displacements of a manipulator from an eye-in-hand camera image. With this method, even in the presence of camera calibration errors, we can formally guarantee that the system is stable and that the entire target remains visible during any visually servoed motion.

This paper focuses on the design of robust motion rate sensor based control of robots. Considering the widely used task function approach of this problem, we first derive a general condition on the control gains with respect to the uncertainties, so as to guarantee bounded tracking errors. This general result is applied to an example case, consisting of a 2-DOF vision based controller. Experimental results of comparing different control strategies are presented.

We use sliding mode control theory to design a 3D vision based controller that is robust to bounded parametric estimation errors. First, a model of an eye-in-hand robotic system is derived and sources of uncertainties are listed. Additionally, bounds on the different uncertainties are discussed and their influence on the overall gain of the system is derived. Due to an appropriate selection of the sliding surface, based on the quaternion representation for rotations, a switching controller is proposed. Six degrees of freedom vision based tracking experiments under weak calibration conditions emphasize the practical efficiency of the algorithm.