Cosserat rod theory proved efficient modeling performances in robotics, especially in the context of continuum robots, in the past decade. The implementation of such theory is far from being unique and straightforward. We consider the illustrative example of multisegment, general routing tendon actuated continuum robots in their nominal static operating regime. This article details two main approaches based on Cosserat rod modeling, namely, the Newtonian and Lagrangian approaches. We provide a walk-through guide regarding theoretical derivations and numerical implementation of both approaches, together with a proof of equivalence. This comparative study is supplemented with novel contributions and extensions of each approach and in-depth discussion of their performances and applicability, as well as highlighting their special features.

Navigation inside luminal organs is an arduous task that requires non-intuitive coordination between the movement of the operator's hand and the information obtained from the endoscopic video. The development of tools to automate certain tasks could alleviate the physical and mental load of doctors during interventions allowing them to focus on diagnosis and decision-making tasks. In this paper we present a synergic solution for intraluminal navigation consisting of a 3D printed endoscopic soft robot that can move safely inside luminal structures. Visual servoing based on Convolutional Neural Networks (CNNs) is used to achieve the autonomous navigation task. The CNN is trained with phantoms and in-vivo data to segment the lumen and a model-less approach is presented to control the movement in constrained environments. The proposed robot is validated in anatomical phantoms in different path configurations. We analyze the movement of the robot using different metrics such as task completion time smoothness error in the steady-state mean and maximum error. We show that our method is suitable to navigate safely in hollow environments and conditions which are different than the ones the network was originally trained on.

Optical Coherence Tomography (OCT) is increasingly used in endoluminal procedures since it provides high-speed and high resolution imaging. Distortion and instability of images obtained with a proximal scanning endoscopic OCT system are significant due to the motor rotation irregularity, the friction between the rotating probe and outer sheath and synchronization issues. On-line compensation of artefacts is essential to ensure image quality suitable for real-time assistance during diagnosis or minimally invasive treatment. In this paper, we propose a new online correction method to tackle both B-scan distortion, video stream shaking and drift problem of endoscopic OCT linked to A-line level image shifting. The proposed computational approach for OCT scanning video correction integrates a Convolutional Neural Network (CNN) to improve the estimation of azimuthal shifting of each A-line. To suppress the accumulative error of integral estimation we also introduce another CNN branch to estimate a dynamic overall orientation angle. We train the network with semi-synthetic OCT videos by intentionally adding rotational distortion into real OCT scanning images. The results show that networks trained on this semi-synthetic data generalize to stabilize real OCT videos, and the algorithm efficacy is demonstrated on both ex vivo and in vivo data, where strong scanning artifacts are successfully corrected.

U reteroscopy is becoming the first surgical treatment option for the majority of urinary affections. This procedure is performed using an endoscope which provides the surgeon with the visual information necessary to navigate inside the urinary tract. Having in mind the development of surgical assistance systems, that could enhance the performance of surgeon, the task of lumen segmentation is a fundamental part since this is the visual reference which marks the path that the endoscope should follow. This is something that has not been analyzed in ureteroscopy data before. However, this task presents several challenges given the image quality and the conditions itself of ureteroscopy procedures. In this paper, we study the implementation of a Deep Neural Network which exploits the advantage of residual units in an architecture based on U-Net. For the training of these networks, we analyze the use of two different color spaces: gray-scale and RGB data images. We found that training on gray-scale images gives the best results obtaining mean values of Dice Score, Precision, and Recall of 0.73, 0.58, and 0.92 respectively. The results obtained shows that the use of residual U-Net could be a suitable model for further development for a computer-aided system for navigation and guidance through the urinary system.

Surgical tool segmentation in endoscopic images is the first step towards pose estimation and (sub-)task automation in challenging minimally invasive surgical operations. While many approaches in the literature have shown great results using modern machine learning methods such as convolutional neural networks, the main bottleneck lies in the acquisition of a large number of manually-annotated images for efficient learning. This is especially true in surgical context, where patient-to-patient differences impede the overall generalizability. In order to cope with this lack of annotated data, we propose a self-supervised approach in a robot-assisted context. To our knowledge, the proposed approach is the first to make use of the kinematic model of the robot in order to generate training labels. The core contribution of the paper is to propose an optimization method to obtain good labels for training despite an unknown hand-eye calibration and an imprecise kinematic model. The labels can subsequently be used for fine-tuning a fully-convolutional neural network for pixel-wise classification. As a result, the tool can be segmented in the endoscopic images without needing a single manually-annotated image. Experimental results on phantom and in vivo datasets obtained using a flexible robotized endoscopy system are very promising.

Lasers are being used in various surgical procedures to remove tissue or bones, to coagulate vessels or other structures. Due to difficulties in handling only a limited number of surgeons manage to display sufficient levels of precision in Minimally Invasive Surgery (MIS) procedures. Prior works on robotic laser surgery demonstrated shorter learning curves and higher ablation precision, but unfortunately ignored the fact that most clinically relevant tasks are bi-manual by nature. Surgeons are also reluctant to use current commercial surgical robotic systems for complex laser tasks, indicating that the lack of haptic feedback prevents them from efficient and safe tissue handling in preparation of laser treatment. This paper expands earlier robotic laser work towards bi-manual operation. The paper introduces a system for precisely tensioning tissue that is being targeted by the laser. An artificial test-setup that captures some essential features of bimanual laser surgery is described. Experiments have been conducted to investigate the effect of haptic feedback on ablation performance. A comparison is made of achievable levels of ablation precision when there is no haptic feedback, when there is haptic feedback and when an automatic tension control algorithm is deployed. The conducted experimental results confirm the great potential of haptic feedback and automatic tensioning systems for complex bi-manual lasering tasks.

A novel fluidic actuation system has been developed for in situ imaging of anatomic tissues. The actuator consists of a micromachined superelastic tool guide driven by a pair of pneumatic artificial muscles. Two additional working channels allow easy interchange of instruments or sensing equipment. This paper describes the design and construction of the actuation system. Experimental results are also reported indicating a bending repeatability of 0.1 degrees and an operational bandwidth exceeding 8Hz. To show-case the performance of the device, the actuator was loaded with an all-optical ultrasound imaging probe. First scanned images of human placental tissue surface using an all-optical ultrasound probe are presented. While a model has been developed to estimate the probe position in space as function of the input pressure, in future work, this model will be complemented with additional sensor measurements of the bending probe taking into account the hysteretic behaviour of both muscles and nitinol structure.

Advances in miniature surgical instrumentation are key to less invasive and safer medical interventions. In cardiovascular procedures interventionalists turn towards catheter-based interventions, treating patients considered unfit for classical more invasive approaches. Improvements in design and steerability of catheters could further reduce the invasiveness of these interventions. For example, by improving controllability and interaction forces with the vessels, tissue damage could be limited. Through improved steerability and coordinated control, operation times and exposure to radiation might also be reduced. Latter argument formed the original motivation for the development of teleoperated robotic catheters. Despite the large kinematic dissimilarity and thus non-trivial mapping between joystick input and catheter output motion, few investigations have been conducted to find intuitive mappings that allow straightforward catheter steering. This paper presents some recent work in this direction. Three promising mappings are proposed. The mappings were implemented and validated upon a robotic catheter moving inside an artificial aorta model. Experimental results show good steerability of the robotic catheter for all the mappings. Although superiority of one mapping with respect to the others was observed, further investigation and validation is planned. In the future, additional visual cues that increase the situational awareness of the user are expected to further simplify the steering.

This paper presents a biomechanical model for tissue deformations in the case of tangential micro-probe scanning for image acquisition. The tissue is modelled as a rigid body and its deformations - considered as elastic - as springs between this body and a fixed reference body. The contact between the probe and the tissue is then considered as a Hertzian sphere-plane contact with a Coulomb friction force. Given those hypotheses, an analytical model of the tissue deformations for 2D tangential movements along the locally planar tissue surface can be established. Similarly to the work of Erden et al. [1], the model has a unique parameter: the loading distance of the tissue. For given scan conditions, this parameter can be calibrated with a simple back-and-forth movement and image measurements. It is of particular interest in minimally invasive surgery where measurements of the friction forces or of the mechanical parameters of the tissue are complex to carry out. Simulations are in accordance with experiments and show that this model allows for accurate estimation of the probe/tissue trajectory in one dimension scans. Moreover, unlike previous studies, the model allows the estimation of the probe/tissue trajectory also for 2D scans. Both coupling behaviour and stick/slip transitions when scanning direction changes are taken into account. However experiments show that anisotropy is an important parameter when studying 2D coupling behaviour. Therefore, an extension of the model that takes anisotropic behaviour of the tissue into account is proposed. Experiments carried out on ex vivo bovine liver and chicken muscle tissues show that the probe/tissue trajectory is accurately predicted by the model. However, this increases the number of parameters to five. As a consequence, unlike in the isotropic case, the parameters can not be simply calibrated using a back-and-forth movement. Further work will be carried out towards finding an easy and effective calibration procedure that applies both to the isotropic and anisotropic cases.

This paper presents the mechanical design of a distal scanner to perform a spiral scan for mosaic-imaging with a confocal microlaparoscope. First, it is demonstrated with ex vivo experiments that a spiral scan performs better than a raster scan on soft tissue. Then a mechanical design is developed in order to perform the spiral scan. The design in this paper is based on a conic structure with a particular curved surface. The mechanism is simple to implement and to drive; therefore, it is a low-cost solution. A 5:1 scale prototype is implemented by rapid prototyping and the requirements are validated by experiments. The experiments include manual and motor drive of the system. The manual drive demonstrates the resulting spiral motion by drawing the tip trajectory with an attached pencil. The motor drive demonstrates the speed control of the system with an analysis of video thread capturing the trajectory of a laser beam emitted from the tip.

This paper presents an approach for understanding the soft tissue behavior in surface contact with a hard object scanning the tissue. The application domain is confocal microlaparoscope imaging, mostly used for imaging the outer surface of the organs in the abdominal cavity. The probe (optic-head) is swept over the tissue to collect sequential images to obtain a large field of view with mosaicing. The problem we address is that the tissue also moves with the probe due to its softness; therefore the resulting mosaic is not in the same shape and dimension as traversed by the probe. Our approach inspires from the finger slip studies and adapts the idea of load-and-slip that explains the movement of the finger when dragged on a hard surface. We propose the concept of loading-distance and perform measurements with in total 84 experiments on beef liver and chicken breast tissues. Our results indicate that the loading-distance can be measured prior to a scan and be used during the scan in order to compensate the movement of the probe. In this way we can have an image-mosaic of the tissue surface in a desired shape.

Probe-based confocal laser endomicroscopy is a recent tissue imaging technology that requires placing a probe in contact with the tissue to be imaged and provides real time images with a microscopic resolution. Additionally, generating adequate probe movements to sweep the tissue surface can be used to reconstruct a wide mosaic of the scanned region while increasing the resolution which is appropriate for anatomico-pathological cancer diagnosis. However, properly controlling the motion along the scanning trajectory is a major problem. Indeed, the tissue exhibits deformations under friction forces exerted by the probe leading to deformed mosaics. In this paper we propose a visual servoing approach for controlling the probe movements relative to the tissue while rejecting the tissue deformation disturbance. The probe displacement with respect to the tissue is firstly estimated using the confocal images and an image registration real-time algorithm. Secondly, from this real-time image-based position measurement, the probe motion is controlled thanks to a simple proportional-integral compensator and a feedforward term. Ex vivo experiments using a Stäubli TX40 robot and a Mauna Kea Technologies Cellvizio imaging device demonstrate the effectiveness of the approach on liver and muscle tissue.

Probe-based confocal laser endomicroscopy is a promising technology for performing minimally-invasive optical biopsies. With the help of mosaicing algorithms, several studies reported successful results in endoluminal surgery. In this paper, we present a prototype for making robotized optical biopsies on a variety of organs inside the abdominal cavity. We chose a macro-micro association, with a macropositioner, a micropositioner and a passive mechanical compensation of physiological motion. The probe is actuated by three hydraulic micro-balloons and can be moved on the surface of an organ to generate a mosaic. This paper presents the design and experimental results of a first in vivo trial on a porcine model.