Polyp dissection requires very accurate detection of the region of interest and high-precision cutting with adequate safety margins. Robot-assisted polyp dissection is a solution to accomplish high-quality intervention. This paper proposes a method to constrain the robot to follow an accurate dissection path based on Virtual Fixtures (VF). The VFs are created via specific control points obtained directly from images of the surgical scene and are updated by the vision algorithm. The VF constraints can autonomously adapt themselves to environment changing during the surgical intervention. The entire pipeline is validated through experiments on the da Vinci Research Kit (dVRK) robot.

International audience

Minimally invasive robotic surgery holds a fundamental role in modern surgery. However, one of its major limitations compared to classic laparoscopy is that the surgeon can only rely on visual perception, for the lack of haptic force feedback. A new solution for a force sensor placed at the end-tip of the trocar is presented here. This solution allows measuring the interaction forces between the surgical instrument and the environment without any changes to the instrument structure and with full adaptability to different robot platforms and surgical tools. A prototype of the sensor has been realized with 3D printed technology for a proof of concept. The static and dynamic characterization of the sensor is provided together with experimental validation.

The da Vinci Research Kit (DVRK) is a telerobotic surgical research platform endowed with an open controller that allows position, velocity and current control. We consider the problem of modelling and identification of both the Patient Side Manipulators (PSMs) and of the Master Tool Manipulators (MTMs) of the platform. This problem is relevant when realistic dynamic simulations have to be performed using standard software tools, but also for the design of model-based control laws, and for the implementation of sensorless strategies for collision detection or contact force estimation. A LMI-based approach is used for the identification of the robot dynamics in order to guarantee the physical feasibility of the parameters that is not ensured by standard least-squares methods. The identified models are validated experimentally.

The problem of image based visual servoing for robots working in a cluttered dynamic environment is addressed in this paper. It is assumed that the environment is observed by depth sensors which allow to measure the distance between any moving obstacle and the robot. Also an eye-in-hand camera is used to extract image features. The main idea is to control suitable image moments and to relax a certain number of robot's degrees of freedom during the interaction phase. If an obstacle approaches the robot, the main visual servoing task is relaxed partially or completely, while the image features are kept in the camera field of view by controlling the image moments. Fuzzy rules are used to set the desired values of the image moments. Beside that, the relaxed redundancy of the robot is exploited to avoid collisions. After removing the risk of collision, the main visual servoing task is resumed. The effectiveness of the algorithm is shown by several case studies on a KUKA LWR 4 robot arm.

This paper presents an experimental study on human-robot comanipulation in the presence of kinematic redundancy. The objective of the work is to enhance the performance during human-robot physical interaction by combining Cartesian impedance modulation and redundancy resolution. Cartesian impedance control is employed to achieve a compliant behavior of the robot's end effector in response to forces exerted by the human operator. Different impedance modulation strategies, which take into account the human's behavior during the interaction, are selected with the support of a simulation study and then experimentally tested on a 7-degree-of-freedom KUKA LWR4. A comparative study to establish the most effective redundancy resolution strategy has been made by evaluating different solutions compatible with the considered task. The experiments have shown that the redundancy, when used to ensure a decoupled apparent inertia at the end effector, allows enlarging the stability region in the impedance parameters space and improving the performance. On the other hand, the variable impedance with a suitable modulation strategy for parameters' tuning outperforms the constant impedance, in the sense that it enhances the comfort perceived by humans during manual guidance and allows reaching a favorable compromise between accuracy and execution time.

This paper addresses the problem of controlling a robot arm executing a cooperative task with a human who guides the robot through direct physical interaction. This problem is tackled by allowing the end effector to comply according to an impedance control law defined in the Cartesian space. While, in principle, the robot's dynamics can be fully compensated and any impedance behaviour can be imposed by the control, the stability of the coupled human-robot system is not guaranteed for any value of the impedance parameters. Moreover, if the robot is kinematically or functionally redundant, the redundant degrees of freedom play an important role. The idea proposed here is to use redundancy to ensure a decoupled apparent inertia at the end effector. Through an extensive experimental study on a 7-DOF KUKA LWR4 arm, we show that inertial decoupling enables a more flexible choice of the impedance parameters and improves the performance during manual guidance.

In this paper, the problem of controlling a robot manipulator in task space, while guaranteeing a compliant behavior for the redundant degrees of freedom, is considered. This issue may arise in the case where the robot experiences an interaction on its body, especially in the presence of humans. The proposed approach guarantees correct task execution and compliance of the robot's body during intentional or accidental interaction in the null space of the main task, simultaneously. The asymptotic stability of the task-space error is ensured by using suitable observers to estimate and compensate the generalized forces acting on the task variables, without using joint torque measurements. Two different controller-observer algorithms are designed, and they are based on the task-space error and on the generalized momentum of the robot, respectively. The performance of the proposed algorithms is verified in experiments on a 7R lightweight robot arm.

The computation of the grasping forces for a multiarm robotic manipulation system (e.g., an anthropomorphic bimanual system) is considered in this paper. This problem is formulated as a convex optimization problem, also considering joint torque constraints. An algorithmic solution suitable for online implementation is presented, which allows a substantial reduction in the computational load by adopting a compact formulation and dynamically decreasing the number of active torque constrains. Moreover, for the case of a bimanual manipulation system, a suboptimal single-hand optimization algorithm is proposed and compared with that providing the optimal solution. Finally, a new algorithm for a valid initial-point evaluation is proposed. The effectiveness of the described methods has been tested in a simulation case study where the grasping forces of a humanoid torso equipped with two five-finger robotic hands are modified online to handle a load with a time-varying mass.

The problem of grasping force optimization for a robotic system equipped with multi-fingered hands is considered in this paper. This problem is cast as a convex optimization problem, considering also joint torque constraints. A solution suitable for on-line implementation is proposed, which allows a substantial reduction of the computational load by dynamically decreasing the number of active torque constraints. Moreover, for the case of a bimanual manipulation system, a sub-optimal single-hand optimization algorithm is presented and compared with the optimal one. The effectiveness of the described methods has been tested in a simulation case study.

In this paper an adaptive multi-priority nonlinear control algorithm for a redundant manipulator system is developed based on the Lyapunov like approach. The method considers the parametric uncertainties in the system and defines a proper filtered error signal to achieve asymptotic stability and convergence in tracking error, both for the main task and sub-tasks according to the allocated priority. The performance of the proposed method is studied by some numerical simulations.

In this paper a new approach for the null-space impedance control of a kinematically redundant robot is proposed. The approach is useful for the case where the robot experience an external interaction on the body, especially in the presence of humans. The proposed algorithm guarantees safe and dependable physical interaction of the robot body with the environment, thanks to the null-space impedance control. At the same time, the correct execution of the task assigned to the end effector is ensured by a disturbance observer. The algorithm does not require joint torque measurements. The performance of the proposed controller is verified through simulations on 7R KUKA lightweight robot arm.

In this paper, a new algorithm for online grasping force optimization (GFO) of a dextrous robotic hand is presented. The GFO problem is cast in a convex optimization problem, considering also torque joint constraints. The proposed formulation allows to simplify the computational complexity of the problem by dynamically reducing the number of active torque constraints. Moreover, differently from other approaches, it does not require the evaluation of a new initial point at the beginning of each iteration. The effectiveness and the performance of the proposed method have been tested in a simulation case study where the hand manipulates a load with time-varying mass.

In this paper, a kinematic model for motion coordination and control of a redundant robotic dual-arm/hand system is derived, which allows to compute the object pose from the joint variables of each arm and each finger as well as from a suitable set of contact variables. This model is used to design a two-stage control scheme to achieve a desired object motion and maintain desired normal contact forces applied to the object. Several secondary tasks are accomplished through a prioritized task sequencing management of the whole system redundancy. A simulation case study is presented to demonstrate the effectiveness of the proposed approach.

This paper presents a dynamic level control algorithm to meet simultaneously multiple desired tasks based on allocated priorities for redundant robotic systems. It is shown that this algorithm can be treated as a general framework to achieve control over the whole body of the robot and some of the previously developed results are formalized using this approach. Null-space impedance control is proposed as one of the main results of using this method and is evaluated by means of computer simulation.

In this paper a method for fast visual grasp of unknown objects with a multi-fingered hand is presented. The algorithm is composed of an object surface reconstruction algorithm and a local grasp planner, evolving in parallel. The former uses an elastic reconstruction surface, whose dimensions are assigned initially by a preshaping process, and which shrinks toward the object until some parts of the surface intercept the object visual hull. The latter moves the fingertips on the current available reconstruction surface towards points which are optimal (in a local sense) with respect to a certain number of indices weighting both the grasp quality and the kinematic configuration of the hand. Experiments are presented, showing the effectiveness of the proposed algorithm.

A new method for fast visual grasp of unknown objects using a camera mounted on a robot in an eye-in-hand configuration is presented. The method is composed of a fast iterative object surface reconstruction algorithm and of a local grasp planner, evolving in a synchronized parallel way. The reconstruction algorithm makes use of images taken by a camera carried by the robot. A reconstruction sphere, virtually placed around the object, is iteratively compressed towards the object visual hull, dragging out the fingers attached to it. Between two steps of the reconstruction process, the planner moves the fingers, floating on the current reconstructed surface, according to suitable quality measures. The fingers keep moving until a local minimum is achieved, then a new object surface estimation provided by the reconstruction process is considered. Quality measures considering both hand and grasp proprieties are adopted. Simulations are presented to show the performance of the proposed algorithm.

In this paper, an approach to interaction control of a robot manipulator with a partially known environment is proposed. The environment is a rigid object of known geometry but of unknown and possibly time-varying position and orientation. An algorithm for online estimation of the object pose is adopted, based on visual data provided by a camera as well as on forces and moments measured during the interaction with the environment. This algorithm is embedded into an impedance control scheme, resulting in a position-based visual impedance control. Experimental results are presented for the case of an industrial robot manipulator in contact with a planar surface.

This paper deals with the problem of position-based visual servoing in a multiarm robotic cell equipped with a hybrid eye-in-hand/eye-to-hand multicamera system. The proposed approach is based on the real-time estimation of the pose of a target object by using the extended Kalman filter. The data provided by all the cameras are selected by a suitable algorithm on the basis of the prediction of the object self-occlusions, as well as of the mutual occlusions caused by the robot links and tools. Only an optimal subset of image features is considered for feature extraction, thus ensuring high estimation accuracy with a computational cost independent of the number of cameras. A salient feature of the paper is the implementation of the proposed approach to the case of a robotic cell composed of two industrial robot manipulators. Two different case studies are presented to test the effectiveness of the hybrid camera configuration and the robustness of the visual servoing algorithm with respect to the occurrence of occlusions

An algorithm for the estimation of the position and orientation of a moving object using a hybrid eye-in-hand/eye-to-hand multi-camera system is presented. Based on the extended Kalman filter, this approach exploits the data provided by all the cameras without "a priori" discrimination, allowing real-time estimation. The proposed formulation can be used with different kinds of image features and different representations of the object orientation. A simulation case study is reported to test the feasibility and the effectiveness of the proposed technique

The problem of interaction control of a robot manipulator with a partially known environment is considered. The environment is a rigid object of known geometry but of unknown and possibly time varying position and orientation. An algorithm for online estimation of the object pose is presented, based on visual data provided by a camera. The estimation accuracy is improved during the interaction by using also force and joint position measurements. The proposed method can be exploited to implement any kind of interaction control strategy. A simulation case study is considered for the case of impedance control of a robot manipulator in contact with a planar surface

In this paper the problem of automatic trajectory planning and inverse kinematics for a robotic system purposely designed to extinguish fires in road and railway tunnels is considered. The robot is composed by a self-cooling monorail vehicle carrying a fire fighting monitor. A fuzzy inference system is adopted for the automatic generation of the task-space trajectory for the robot and to solve the inverse kinematics problem in the presence of redundant degrees of freedom. Redundancy also allows assigning additional tasks besides the primary task. Simulation case studies are presented to test the performance of the whole system in a typical intervention scenario.

An environment for open real-time control of an industrial robotic cell is presented in this paper. The experimental setup is composed of two industrial robot manipulators equipped with force/torque sensors and pneumatic grippers, a vision system and a belt conveyor. The original industrial robot controllers have been replaced by a single PC with software running under a real-time variant of the Linux operative system. The new control environment allows advanced control schemes to be developed and tested for the single robots and for the dual-arm robotic cell, including force control and visual servoing tasks. An advanced user interface and a simulation environment have been developed, which permit fast, safe and reliable prototyping of planning and control algorithms.

In this paper an observer-based fault diagnosis (FD) approach for autonomous underwater vehicles (AUVs), subject to actuator faults (i.e., faults affecting the propulsion system and/or the control surfaces), is proposed. A diagnostic observer is developed based on the available dynamic model of the AUV. Compensation of unknown dynamics, uncertainties and disturbances is achieved through the adoption of a class of neural interpolators (support vector machines, SVMs) trained off line. On the other hand, interpolation of unknown actuator faults is performed by adopting a radial basis function (RBF) network, whose weights are adaptively tuned on line. The effectiveness of the approach is tested in a simulation case study developed for the NPS AUV II (PHOENIX) vehicle.

An algorithm for the visual estimation of the pose of a moving object is presented in this paper. The algorithm exploits the prediction capability of the extended Kalman Filter to realize in real time a dynamic optimal selection of the object image features used for pose estimation. The robustness of the system with respect to the measurement noise and modelling errors is enhanced by using an adaptive scheme. Experimental case studies are presented to prove the effectiveness of the proposed approach.

The problem of tracking the position and orientation of a moving object using a stereo camera system is considered in this paper. A robust algorithm based on the extended Kalman filter is adopted, combined with an efficient selection technique of the object image features, based on Binary Space Partitioning tree geometric models. An experimental study is carried out using a vision system of two fixed cameras.

The problem of visual tracking of multiple objects is considered in this paper. Special emphasis is devoted to the case when two or more objects overlap with respect to the visual system causing occlusion. The algorithm is based on the Kalman filtering and binary space partition tree representations of the objects geometry. The real-time implementation of the algorithm is experimentally tested for the case of visual tracking of two objects using two cameras.

In this paper a computationally efficient algorithm for real time estimation of the position and orientation of moving objects from visual measurements of a system of fixed cameras is proposed. The algorithm is based on extended Kalman filtering of the measurements of the position of suitable feature points selected on the target objects. The effectiveness of the algorithm is improved by using a pre-selection method of the feature points which takes advantage of the Kalman filter prediction capability combined with a BSP tree modeling technique of the objects geometry. Computer simulations are presented to test the performance of the estimation process in the presence of noise, different types of lens geometric distortion, quantization and calibration errors.

The problem of real-time pose estimation of moving objects using a stereo video camera system is considered in this paper. A computationally efficient algorithm is proposed based on Kalman filtering of the position measurements of suitable feature points selected on the target objects. The efficiency of the algorithm is improved by adopting a new pre-selection technique of the feature points, based on binary space partition (BSP) trees, which takes advantage of the Kalman filter prediction capability. Computer simulations are presented.

The problem of controlling force and position of a flexible link manipulator in contact with a compliant environment is considered in this paper. Using the singular perturbation theory, a slow subsystem associated with rigid motion dynamics and a fast sub-system associated with link flexible dynamics are identified. Hence, a two-time scale control strategy is adopted, consisting of a force and position control action for the slow sub-system and a stabilizing action for the fast subsystem. Both regulation and tracking problems are considered for the position, while the contact force is kept to a constant desired value. Simulations are presented for a two-link planar manipulator under gravity.

The goal of the paper is the application of a geometrically consistent impedance concept to control interaction with the environment of a rigid object manipulated by a dual-robot system. A six-DOF impedance is specified at the object level to confer a compliant behavior for both the translational and the rotational motion when an external force and moment occurs at the contact. Geometric consistency is ensured thanks to the use of the unit quaternion to describe object frame orientation. The resulting object motion is decomposed into the equivalent motions at the end effectors of the two robots, via a task-oriented formulation. The control scheme is derived according to an inverse dynamics strategy with adoption of an inner motion loop providing robustness to unmodeled dynamics and disturbances. Experimental results on the two industrial robots available in the lab are discussed.

Focuses on the problem of tracking a desired end-effector position and orientation for a robot manipulator. A nonminimal parameterization of the orientation-the unit quaternion-is used to design the control law. Tracking control typically requires full-state measurements, i.e., end-effector position and orientation as well as linear and angular velocities. However, while position and orientation measurements are available, often velocities have to be estimated. In the paper, an observer-controller scheme is proposed which ensures tracking in the task space without using velocity measurements. The performance of the proposed scheme is evaluated both in simulation and through experimental tests on an industrial robot.

This work is focused on impedance control of redundant manipulators. A spatial impedance formulation is presented where general 6-DOF end-effector tasks can be handled. A singularity-free angle/axis representation of end-effector orientation is used which allows geometric task consistency to be preserved. An inverse dynamics control with a dynamically consistent inverse of the geometric Jacobian matrix is developed with the adoption of an inner loop acting on the end-effector position and orientation conferring robustness to unmodeled dynamics and external disturbances. Stabilization of null-space velocities is ensured and utilization of redundant degrees of mobility is carried out to optimize an additional task function. Experimental results on a seven-joint industrial robot with force/torque sensor are discussed.

A new approach to 6-DOF impedance control is proposed, where the end-effector orientation displacement is derived from the rotation matrix expressing the mutual orientation between the compliant frame and the desired frame. An alternative Euler angles-based description is proposed which mitigates the effects of representation singularities. Then, a class of angle/axis representations are considered to derive the dynamic equation for the rotational part of a 6-DOF impedance at the end effector, using an energy-based argument. The unit quaternion representation is selected to further analyze the properties of the rotational impedance. The resulting impedance controllers are designed according to an inverse dynamics strategy with contact force and moment measurements, where an inner loop acting on the end-effector position and orientation error is adopted to confer robustness to unmodeled dynamics and external disturbances. Experiments on an industrial robot were carried out, and the results of case studies are discussed.

The work is aimed at studying the problem of controlling the moment and the orientation of a robot manipulator whose end effector is in contact with a compliant environment. Differently from classical operational space formulations, a geometrical approach is pursued where orientation displacements are described in terms of unit quaternions. Regulation to a constant desired moment and tracking of a time-varying desired orientation trajectory is achieved, and the convergence of the closed-loop system is analyzed. Simulation results for a six-joint industrial robot are developed and an experimental test is carried out to demonstrate the effectiveness of the proposed approach in a practical contact task.

In the framework of parallel force/position control for a robot manipulator in contact with a compliant environment, a new scheme is proposed which is aimed at controlling the end-effector force in the face of uncertainty on the surface stiffness. The controller is of inverse dynamics type with a force feedforward action. Adaptation to unknown stiffness is achieved by resorting to a suitable estimate update law driven by the force error. Tracking of both position along the unconstrained directions and force along the constrained direction is ensured. Experimental results on an industrial robot with open control architecture are presented.

The problem of regulation of force and position for robot manipulators in contact with an elastically compliant plane is considered in this work. As an improvement over a recently proposed regulator, a new control scheme is presented which does not require velocity measurements due to a suitable filtering action. Local asymptotic stability is proven via a Lyapunov argument. A case study is developed to illustrate the performance of the scheme on an industrial robot.

The problem of designing a control scheme for robot manipulators in contact with a compliant surface is considered. A passivity-based force/position control scheme is designed. A keen choice of the reference vector used in the control law is proposed as a function of the end-effector position and velocity errors and the integral of the contact force error. It is demonstrated that the scheme ensures tracking of the unconstrained components of the desired end-effector trajectory with regulation of the desired contact force along the constrained direction. In the imperfect model compensation case, the scheme is made adaptive with respect to a set of dynamic parameters. Numerical case studies are presented for an industrial robot manipulator.>