In application domains such as surgical robotics, fully autonomous control remains a long-term ambition and the systems are mostly teleoperated. In this article, the presence of an operator in-the-loop is exploited to perform the online registration of an initially inaccurate haptic guidance and the calibration of robot kinematic models using operator’s intention instead of relying on exteroceptive sensors. This is used to improve online haptic guidance in the context of shared control, or to progress toward automatic task completion after an initial learning phase. The method presented in this article is based on an optimization in the task space to minimize the errors between the executed and desired trajectories, both estimated from models. This approach is particularly relevant when the execution of a planned task would suffer from errors that exteroceptive measurements could not fully correct, because of sensor inaccuracy or unavailability. A user study realized for a drawing task is detailed to illustrate that initially inaccurate task registration and robot models can be corrected from user inputs only. The results show that the proposed algorithm can learn the correct models, which in turns significantly improves the quality of the haptic guidance and decreases path deviations during the teleoperated task.

In this paper, a method for 3D non-rigid motion estimation of a surface using an RGB-D camera in eye-in-hand configuration is presented. The eye-in-hand configuration eliminates errors typically associated with camera-end-effector calibration, and is thus desirable for task on moving surfaces such as bioprinting. However, its implementation is challenging since camera and surface of interest are moving, making mesh-based approaches unsuitable. Thus, the proposed method operates directly on point clouds, benefiting from accurate and simplified data processing. A point cloud contains both intensity and depth data, with the former used to estimate in-plane deformation and the latter to compute full 3D deformation. Surface deformation is modeled via a Thin Plate Spline model. The method accuracy is assessed at 0.1 mm accuracy in simulated datasets, rendering it suitable for precision tasks, and its feasibility is validated experimentally on a moving platform that deforms at a rate of 0.8 Hz with a 4 mm in-plane amplitude and a 20 mm elevation amplitude.

Haptic guidance can improve accuracy and dexterity during the teleoperation of a robot, but only if the model of the task used to provide the assistance is accurate. In medical robotics, the registration of a task from pre-operative planning from medical images to the robot’s task-space can be erroneous. Additionally, the deformability of the environment can require online correction of a planned task. Therefore, we propose a method to update the geometry and the registration of a pathfollowing task online. This model is simultaneously used to physically guide the user during the teleoperation. Experimental results obtained on a haptic interface show the validity of the approach for a simulated 2D task.

Electromyogram (EMG) signal has been long used in human-robot interface in literature, especially in the area of rehabilitation. Recent rapid development in artificial intelligence (AI) has provided powerful machine learning tools to better explore the rich information embedded in EMG signals. For our specific application task in this work, i.e. estimate human finger force based on EMG signal, a LightGBM (Gradient Boosting Machine) model has been used. The main contribution of this study is the development of an objective and automatic optimal feature selection algorithm that can minimize the number of features used in the LightGBM model in order to simplify implementation complexity, reduce computation burden and maintain comparable estimation performance to the one with full features. The performance of the LightGBM model with selected optimal features is compared with 4 other popular machine learning models based on a dataset including 45 subjects in order to show the effectiveness of the developed feature selection method.

Needle placement errors can mitigate the effectiveness of the diagnosis or the therapy, sometimes with catastrophic outcomes. Previous design of a simplified model for needle deflection estimation was motivated by the clinical constraints of ARCS (Abdomino-pelvic Robotic-driven slightly flexible needle insertion performed in CT/MRI-guided Scenario). We present in this work, the validation results for the needle deflection prediction model. Its robustness is evaluated under an unknown context such as a different robotic platform, facing uncertainties conditions not conceived previously in the model's confection. In addition, the work presents the development and validation experiments of an adaptive path planner that uses the model as predictor's strategy. It provides pre-operative planning assistance, as well as intra-operative decision-making support. The experiments results showed average error around 1mm for the pre-operative planning and the intra-operative replanning approach showed to be very robust to correct the initial predictions, showing average error smaller than 1 mm.

In the medical field, several surgical simulators and training platforms have been developed to help novice surgeons improve their surgical skills, as well as perform preoperative planning. In this paper, a haptic platform for surgical needle insertion training gestures is presented. Instead of relying on active actuators, an alternative solution, consisting in passive brakes based on Electro-Rheological (ER) fluids, is proposed, to provide a safe and realistic physical feedback to the physician. This platform generates a passive repulsive force against the user's movement, providing him/her a physical stimulus and, thus, a realistic haptic feedback. The goal of this project is to prove the reliability of ERF-based brakes to simulate the physical resistance of soft tissues against the movement of a surgical needle, in order to train unskilled practitioners in different scenarios. To achieve this objective, a prototype has been built, its kinematic model has been obtained and experimentally validated. The modelling, the bandwidth analysis and the force control scheme of the platform are also presented.

Controlling the interaction between the robot and living soft tissues has became an important issue as the number of robots inside the operating room increases. Many research works have been done in order to control this interaction. Nowadays, researches are running in force control for helping surgeons in medical procedures such as motion compensation in beating heart surgeries and tele-operation systems with haptic feedback. The viscoelasticity property of the interaction between organ tissue and robotic instrument further complicates the force control design which is much easier in other applications by assuming the interaction model to be elastic (industry, stiff object manipulation, etc.). In order to increase the performance of a model based force control, this work presents a force control scheme using Active Observer (AOB) based on a viscoelastic interaction model. The control scheme has shown to be stable through theoretical analysis and its performance was evaluated and compared with a control scheme based on a classical elastic model through experiments, showing that a more realistic model can increases the performance of the force control.

An analysis of stability and transparency of a force feedback teleoperation system for cutting-edge robotic surgery is presented. Previous works in teleoperated robotic surgery do not consider the real behavior of the environment, which was supposed to be only elastic. However, new surgical procedures in which the environment dynamics plays a crucial role start emerging as a result of technological progress. In robotic assisted beating-heart surgery, for instance, the dynamics of the contact between surgical tools and soft tissues has an impact not only in the performance of the force control task but also in the performance of the teleoperation control scheme in terms of transparency and stability. Therefore, a more realistic description of the environment has to be adopted in order to safely operate during robot-patient interaction. For this purpose, a viscoelastic contact model is introduced into the bilateral teleoperation scheme, and a performance study is provided. The obtained results show the advantages of the selected approach when targeting teleoperated surgical interventions in which the interaction dynamics has become a significant issue.

This paper presents an adaptive approach for 2D motion planning of steerable needles. It combines duty-cycled rotation of the needle with the classic Rapidly-Exploring Random Tree (RRT) algorithm to obtain fast calculation of feasible trajectories. The motion planning is used intraoperatively at each cycle to compensate for system uncertainties and perturbations. Simulation results demonstrate the performance of the proposed motion planner on a workspace based on ultrasound images.

The proposed robotic system provides the surgeon with an augmented sensation of the interaction forces between the instrument and the organ. Such a system aims at increasing the surgeon's dexterity for tasks requiring that only small forces be applied on the organ (eg. for micro-surgery). In the proposed setup, the surgeon manipulates a handle mounted on the instrument. This is a comanipulation system because the surgeon and the robot simultaneously manipulate the instrument. The proposed control scheme allows an augmented force control: the control law ensures that the instrument applies on the organ the same forces that the surgeon applies on the handle but decreased by a scale factor. As a consequence, the forces sensed by the surgeon are the forces between the instrument and the organ amplified by a scale factor. This control scheme is proved stable thanks to a passivity study. Indeed, passivity analysis is a useful tool for the stability analysis of a robot interacting with the environment. Experimental results are presented on a robot dedicated to minimally invasive surgery.

This paper presents a force feedback control scheme for the compensation of periodic motions of organs induced by respiration or heartbeat in minimally invasive robotized surgery. It applies surgical tasks involving a contact between an instrument and a moving organ. It is well known that conventional force control allows for compensating the motion of the environment thanks to its natural disturbance rejection capabilities. However, as experimentally evidenced in the first part of this paper, bandwidth limitations do not allow for exact disturbance rejection. Therefore, in addition to a conventional inner force feedback control loop, an outer control loop based on iterative learning control (ILC) is implemented. It is aimed at compensating the physiological motions, based on the hypothesis that the disturbance is periodic. The transient performances of this ILC controller are improved thanks to a wavelet transform-based approach and conclusive experiments are finally presented, evidencing that the tracking performance under cyclic disturbances is significantly improved.

In this article, the problem of force feedback control of kinematically constrained manipulators (KCMs) is considered. For these robots, we show that the force component selection approach is not appropriate in general to solve the force control problem. Moreover, by formulating the problem in the joint space, we show how to properly design a stable force controller for KCMs subject to arbitrary external forces applied to their end-effector. Experimental results with a kinematically constrained laparoscopic comanipulator illustrate these propositions

Minimally invasive surgery (MIS) challenges the surgeon's skills due to his separation from the operation area, which can be reached with long instruments only. Therefore, the surgeon loses access to the manipulation forces inside the patient. This reduces his dexterity when performing the operation. A new compact and lightweight robot for MIS is presented which allows for the measurement of manipulation forces. The main advantage of this concept is that no miniaturized force sensor has to be integrated into surgical instruments and inserted into the patient. Rather, outside the patient a standard sensor is attached to a modified trocar, which allows for the undisturbed measurement of manipulation forces. This approach reduces costs and sterilizability demands. Results of first force control experiments are presented to show the feasibility of the concepts.