In robotic teleoperation, it is crucial to be able to dynamically adjust interactions with the environment. Drawing inspiration from human behavior during interactions, Variable Impedance Control (VIC) has been widely adopted to enhance robotic flexibility and adaptability. However, maintaining the passivity of such control systems remains a critical safety concern. This paper introduces an optimization-based framework for passive variable impedance control in bilateral teleoperation, combining the advantages of Passivity Filters (PFs), Time-Domain Passivity (TDP) control, and Passive-Set-Position-Modulation (PSPM). The method solves an optimization problem aimed at dissipating the energy that could lead to a lack of passivity. The proposed method is assessed through experiments, illustrating its ability to keep the teleoperation system passive and safe under a variable impedance profile.

To reduce the radiation dose, sparse-view computed tomography (CT) reconstruction has been proposed, aiming to recover high-quality CT images from sparsely sampled sinogram. To eliminate the artifacts present in sparse-view CT images, a new dual-domain diffusion model (DDDM) is proposed, which is composed of a sinogram upgrading module (SUM) and an image refining module (IRM) connected in series. In the sinogram domain, a novel degrading and upgrading framework is defined, in which SUM is trained to upgrade sparse-view sinograms step by step to reverse the degradation process of CT images caused by successive down-sampling of scanning views. In the image domain, IRM adopts an improved denoising diffusion framework to further reduce remaining artifacts and restore image details, where a skip connection from the original sparseview sinogram is introduced to constrain the generation of details. Our DDDM shows significant improvement over deep-learning baseline models in both classical similarity metrics and perceptual loss, and has good generalization to untrained organs. We release our code at https://github.com/YC-Markus/code-for-DDDM.

In this work, we use recent developments in the field of adaptive robust Model Predictive Control (MPC) to build a controller for bilateral teleoperation systems. To guarantee robust constraint satisfaction, we incorporate polytopic tube controllers in the MPC design. In addition, we use online learning methods to learn the environment model. Namely, we use set membership learning to learn the parametric uncertainty bounds and reduce the conservatism of the robust controller, and we combine it with least mean square method to learn a point estimate of the model parameters, which enhances the controller performance. Our simulation demonstrates the effectiveness of the proposed approach in maintaining robust constraint satisfaction and enhancing performance by learning during teleoperation tasks.

In past decades, much progress has been obtained in vision-based robot control theories with traditional image processing methods. With the advances in deep-learning-based methods, convolutional neural network (CNN) has now replaced the traditional image processing methods for object detection and recognition. However, it is not clear how the CNN-based methods can be integrated into robot control theories in a stable and predictable manner for object detection and tracking, especially when the aspect ratio of the object is unknown and also varies during manipulation. In this article, we develop a vision-based control method for robots with an eye-in-hand configuration, which can be directly integrated with existing CNN-based object detectors. The task variables are generated based on parameters of the bounding box from the output of any real-time CNN object detector such as you only look once (Yolo). To address the chattering problem of bounding box, long short-term memory (LSTM) is used to provide smoothed bounding box information. A vision-based controller is then proposed following task-space motion control design formulation in order to keep the object of unknown aspect ratio in the center of field of view of the camera. The stability of the overall closed-loop control system is analyzed rigorously using the Lyapunov-like approach. Experimental results are presented to illustrate the performance of the proposed CNN-based robot controller.

In order to obtain a high-precision contact model that can properly describe the target soft tissue, this paper proposes a hybrid soft contact model based on a clustering algorithm ART-II, which selects the most suitable soft contact model according to the surgical environment. The least-square method is used to identify the parameters of the model online. In the experiments, different parts of animal tissues were used as the experimental objects. The hybrid model was used to identify and switch for the most appropriate soft contact model when dealing with a certain type of animal tissue. The performance of the hybrid model on force estimation was compared with several individual soft contact models. The results showed that the estimated/reconstructed force of the hybrid model was closer to the ground truth measured by the force sensor. In addition, a new reference soft contact model has been purposely added online to verify the expandability of the hybrid model.

International audience

In robot-assisted cardiac surgery, predicting heart motion can help improve the operation accuracy and safety of surgical robots. Different from the conventional prediction schemes which model the point of interest (POI) with only temporal correlation of past observations, this paper proposes an LSTM-based method by exploiting the spatio-temporal correlation of the 3D movements of POI and auxiliary points (APs) on the same surface of the heart. Three different LSTM models are investigated. The first two models define the POI prediction as a pure time-series forecasting problem based on past POI trajectory, and the third model combines the past observations of POI and new observations of APs to take into consideration the extra spatial correlations for prediction. Experimental comparison studies based on 3D coordinates obtained from real stereo-endoscopic videos demonstrate the superior performance of the proposed spatio-temporal LSTM model.

In dynamic minimally invasive surgery environments, 3D reconstruction of deformable soft-tissue surfaces with stereo endoscopic images is very challenging. A simple self-supervised stereo reconstruction framework is proposed to address this issue, which bridges the traditional geometric deformable models and the newly revived neural networks. The equivalence between the classical thin plate spline (TPS) model and a single-layer fully-connected or convolutional network is studied. By alternating training of two TPS equivalent networks within the self-supervised framework, disparity priors are learnt from the past stereo frames of target tissues to form an optimized disparity basis, on which disparity maps of subsequent frames can be estimated more accurately without sacrificing computational efficiency and robustness. The proposed method was verified on stereo-endoscopic videos recorded by the da Vinci®surgical robots.

Optical tweezers have received significant attention in the past few decades and many techniques have been proposed to achieve assorted manipulation tasks on cells or micro-objects. While traditional techniques for optical tweezers utilize laser beams for direct trapping and manipulation of cells, several approaches have also been proposed to achieve grasping and manipulation of cells. Most grasping approaches, however, have failed to solve the existing difficulties in the grasping process of cells due to limited sensing capability and presence of Brownian perturbation in the micro world. In this paper, a novel human-assisted grasping technique for manipulation of biological cells is proposed to facilitate the grasping process of biological cells. In the proposed technique, several microbeads are first trapped and used as fingertips to perform a grasping task on a biological cell. Based on human observation, the positions of the laser beams that trap the microbeads are remotely and simultaneously controlled through a touchscreen, and thus generating a grasping formation of the trapped microbeads to grasp the cell. After the cell has been grasped, a simple region control technique is utilized for automated manipulation of the grasped cell. This paper offers a flexible, reliable and efficient approach for grasping and manipulation of biological cells, and thus extending the feasible applications of optical tweezers. The stability of the control system for manipulation of the grasped cell is investigated by using a Lyapunov approach, and the feasibility of the proposed human-assisted grasping and manipulation technique is demonstrated by experiment result.

Electromyogram (EMG) signal has been long used in human-robot interface in literature, especially in the area of rehabilitation. Recent rapid development in artificial intelligence (AI) has provided powerful machine learning tools to better explore the rich information embedded in EMG signals. For our specific application task in this work, i.e. estimate human finger force based on EMG signal, a LightGBM (Gradient Boosting Machine) model has been used. The main contribution of this study is the development of an objective and automatic optimal feature selection algorithm that can minimize the number of features used in the LightGBM model in order to simplify implementation complexity, reduce computation burden and maintain comparable estimation performance to the one with full features. The performance of the LightGBM model with selected optimal features is compared with 4 other popular machine learning models based on a dataset including 45 subjects in order to show the effectiveness of the developed feature selection method.

Due to the lack of transparent and friendly human-robot interaction (HRI) interface, as well as various uncertainties, it is usually a challenge to remotely manipulate a robot to accomplish a complicated task. To improve the teleoperation performance, we propose a new perception mechanism by integrating a novel learning method to operate the robots in the distance. In order to enhance the perception of the teleoperation system, we utilize a surface electromyogram signal to extract the human operator's muscle activation. As a response to the changes in the external environment, as sensed through haptic and visual feedback, a human operator naturally reacts with various muscle activations. By imitating the human behaviors in task execution, not only motion trajectory but also arm stiffness adjusted by muscle activation, it is expected that the robot would be able to carry out the repetitive tasks autonomously or uncertain tasks with improved intelligence. To this end, we develop a robot learning algorithm based on probability statistics under an integrated framework of the hidden semi-Markov model (HSMM) and the Gaussian mixture method. This method is employed to obtain a generative task model based on the robot's trajectory. Then, Gaussian mixture regression based on HSMM is applied to correct the robot trajectory with the reproduced results from the learned task model. The execution procedures consist of a learning phase and a reproduction phase. To guarantee the stability, immersion, and maneuverability of the teleoperation system, a variable gain control method that involves electromyography (EMG) is introduced. Experimental results have demonstrated the effectiveness of the proposed method.

Most of existing works on the tubes design optimization of concentric-tube robot (CTR) do not include the elastic stability in the optimization criteria. The only work which formulates the elastic stability in the objective function is based on scalarization method which is used in existing multi-objective design optimization. The objective function is formed by a set of weighted objective functions. The selection of the weights is crucial as the optimization results are greatly affected by them and could be misleading if these weights are improperly chosen. As an alternative optimization technique, we use Pareto grid-searching method to avoid this problem and allow a straightforward interpretation of the results following the selection criteria for the parameters to be optimized. This paper shows a three-tube CTR design based on Pareto grid-searching method in order to optimize the reachability and elastic stability of the CTR within a specific curvature range dedicated to the deep anterior brain tumor removal surgery.

In this paper we propose a modeling method for the interaction impedance of a remote soft tissue that contains quasi-periodic physiological motion disturbance. Through this study, it is shown that the interaction with such environment is not passive and its influence should be considered in the teleoperator design. The interaction impedance depends not only on the soft tissue impedance but also on the relationship between the robotic tool motion and the soft tissue motion disturbance. An illustrative case study is presented to demonstrate how to analyze the environment interaction impedance in real application.

Many medical applications can benefit from the new technology of concentric-tube robot (CTR) due to its miniature size, superior steerability, and controllability of the end tool. However, the kinematic modeling of CTR is challenging because of complicated physical phenomena caused by the elasticity interaction between tubes. Existing control methods of CTR are based on inverse kinematics calculation and hence the control performance largely relies on the accuracy of kinematics model used. In this work, we propose a new control method from the actuator level and show that the control design of actuator input in task-space with approximate Jacobian matrix provides more flexibility and robustness in handling inaccuracy in kinematics model. It is shown through simulation study that the proposed control method presents better performance compared with traditional inverse kinematics based control method in face of kinematics inaccuracy.

Tele-surgery has been more and more popular in robotassisted medical intervention. Most existing teleoperation architectures for medical applications adopt 2-channel architectures. The 2-channel architectures have been evaluated in literature and it is shown that some architectures, e.g. position-force (P-F), are able to provide the surgeon a reliable haptic sense of the working environment (transparency). However, stability of these P-F architecture is still a considerable concern especially when physiological disturbances exist in the remote environment. P-PF architecture is proved to provide a convenient alternative. With one more channel 3-channel teleoperation architectures present promising options due to their augmented design flexibility. This paper evaluates stability and transparency of general 3-channel bilateral teleoperation control architectures and provides a design framework guidelines to improve the architectures' stability robustness and optimize the transparency. Simulation evaluations are provided to illustrate how the optimal 3-channel teleoperation architecture is chosen for medical applications given their dedicated requirements.

Controlling the interaction between the robot and living soft tissues has became an important issue as the number of robots inside the operating room increases. Many research works have been done in order to control this interaction. Nowadays, researches are running in force control for helping surgeons in medical procedures such as motion compensation in beating heart surgeries and tele-operation systems with haptic feedback. The viscoelasticity property of the interaction between organ tissue and robotic instrument further complicates the force control design which is much easier in other applications by assuming the interaction model to be elastic (industry, stiff object manipulation, etc.). In order to increase the performance of a model based force control, this work presents a force control scheme using Active Observer (AOB) based on a viscoelastic interaction model. The control scheme has shown to be stable through theoretical analysis and its performance was evaluated and compared with a control scheme based on a classical elastic model through experiments, showing that a more realistic model can increases the performance of the force control.

An analysis of stability and transparency of a force feedback teleoperation system for cutting-edge robotic surgery is presented. Previous works in teleoperated robotic surgery do not consider the real behavior of the environment, which was supposed to be only elastic. However, new surgical procedures in which the environment dynamics plays a crucial role start emerging as a result of technological progress. In robotic assisted beating-heart surgery, for instance, the dynamics of the contact between surgical tools and soft tissues has an impact not only in the performance of the force control task but also in the performance of the teleoperation control scheme in terms of transparency and stability. Therefore, a more realistic description of the environment has to be adopted in order to safely operate during robot-patient interaction. For this purpose, a viscoelastic contact model is introduced into the bilateral teleoperation scheme, and a performance study is provided. The obtained results show the advantages of the selected approach when targeting teleoperated surgical interventions in which the interaction dynamics has become a significant issue.

This paper proposes a novel control strategy for autonomous underwater vehicles (AUVs), named as path tracking, which combines the conventional path following and trajectory tracking control in order to achieve smooth spatial convergence and tight temporal performance as well. This idea is inspired by the previous work of Hindman [1] and Encarnacao [2], however, the path tracking design herein goes from path following to trajectory tracking, which indeed is an inverse way from the previous solutions so that the complex projection algorithm resulting in a local stability is avoided. A kinematics controller is first derived by using Lyapunov direct method where a virtual path parameter is introduced to bring an extra control degree of freedom, and then it is extended to the dynamics of AUVs based on backstepping technique. The resulting nonlinear control design is formally shown and it yields global asymptotic convergence of the AUV to the path. Finally, simulation results illustrate the efficiency of the path tracking control design for AUVs.

In this paper, the regulation problem of rigid-link electrically-driven (RLED) robotic manipulators with uncertain kinematics and dynamics is addressed. A task-space Saturated-Proportional Integral and Differential (SP-ID) based control approach is proposed using backstepping technique to deal with the uncertainties in actuator dynamics, robot dynamics and kinematics. The proposed method is structurally simple and easy for implementation. Sufficient conditions for choosing the feedback gains, approximate Jacobian matrix and motor torque constant matrix are provided to guarantee system stability. Simulation results demonstrate the effectiveness of the proposed approach.

With the advent of new applications in cardiac robotic-assisted minimally invasive surgery (MIS), a demand for the design of efficient motion compensation systems was created. In this context, vision-based techniques seem to be a practical way to retrieve the motion of the beating heart since they do not require the introduction of additional sensors in the limited workspace. In this paper, we propose an efficient method for tracking the heart surface which incorporates two novelties. The first is a thin-plate splines (TPS) parametric model for the heart surface deformation that allows us to better track regions of the heart surface with little texture information which undergo large non-rigid deformations. The second novelty is the incorporation of a performing illumination compensation algorithm to cope with arbitrary illumination changes and increase tracking robustness.We also extend this framework for 3D tracking, to enable full compensation of the heart motion. Extensive experiments conducted onin-vivoandex-vivoheart images attest the notable performance of the algorithm.

In this paper, a vision based tracking controller with adaptation to uncertainty in depth information is presented. Depth uncertainty plays a special role in visual tracking as it appears nonlinearly in the overall Jacobian matrix and hence cannot be adapted together with other uncertain kinematic parameters. We propose a novel parameter update law to update the uncertain parameters of the depth. It is proved that system stability can be guaranteed for the visual tracking task in presence of uncertainties in depth information, robot kinematics and dynamics. Simulation results are presented to illustrate the performance of the proposed controller.

Joint flexibility is an important factor to consider in the robot control design if high performance is expected for the robot manipulators. The research work on control of rigid-link flexible-joint (RLFJ) robot in the literature has assumed that the kinematics of the robot is known exactly. There have been no results so far that can deal with the kinematics uncertainty in RLFJ robot. In this paper, we present the first study on this problem and propose an adaptive regulation method which can deal with the kinematics uncertainty and uncertainties in both link and motor dynamics of the RLFJ robot system. An observer is designed to avoid the use of acceleration due to the fourth-order overall dynamics. Sufficient conditions are derived to guarantee the asymptotic stability of the closed-loop system. Simulation result illustrates the effectiveness of proposed control method

This paper presents a task-space saturated-proportional, integral and differential (SP-ID) regulation approach for robot manipulators with uncertain kinematics and actuator model. The proposed approach is computationally efficient and easy to implement due to its simple structure. It's interesting to observe that in this paper the simple PID type controller is shown not only capable of compensating unknown gravity force, as has been known for long in robot control literature, but also capable of dealing with uncertainties in robot kinematics and actuator model. Sufficient conditions to guarantee system stability are provided and simulation results are presented to show the performance of proposed control method

Most research so far on trajectory tracking control of robot has assumed that the kinematics of the robot is known exactly. This paper extends our recent work on adaptive Jacobian tracking control by deriving a new algorithm for trajectory tracking of robots with uncertain kinematics and dynamics. The algorithm requires only to measure the end-effector position in visual space, besides the robot’s joint angles and joint velocities. Experimental results are presented to illustrate the performance of the proposed controllers. In the experiments, we demonstrate that the robot’s shadow can be used to control the robot.

In this paper, the adaptive regulation problem of rigid-link electrically driven (RLED) robotic manipulators with uncertain kinematics is addressed. A new task-space control scheme is proposed to overcome the uncertainties in actuator dynamics, robot dynamics and kinematics. By using a novel adaptive regressor, we avoid the overparameterization problem which is often met in the adaptive control problem with uncertain actuator model. Sufficient conditions for choosing the feedback gains, approximate Jacobian matrix are provided to guarantee system stability. Simulation results are presented to verify the effectiveness of the proposed control scheme.

Research so far on trajectory tracking control of robot has assumed that the kinematics of the robot is known exactly. In this paper, a new approximate Jacobian adaptive controller is proposed for trajectory tracking of robot with uncertain kinematics and dynamics. It is shown that the robot end effector is able to converge to a desired trajectory with the uncertain kinematics and dynamics parameters being updated online by parameter update laws. Experimental results are presented to illustrate the performance of the proposed controllers.