In this article, a deep learning method for the shape sensing of continuum robots based on multicore fiber bragg grating (FBG) fiber is introduced. The proposed method, based on an artificial neural network (ANN), differs from traditional approaches, where accurate shape reconstruction requires a tedious characterization of many characteristic parameters. A further limitation of traditional approaches is that they require either multiple fibers, whose location relative to the centerline must be precisely known (calibrated), or a single multicore fiber whose position typically coincides with the neutral line. The proposed method addresses this limitation and, thus, allows shape sensing based on a single multicore fiber placed off-center. This helps in miniaturizing and leaves the central channel available for other purposes. The proposed approach was compared to a recent state-of-the-art model-based shape sensing approach. A two-degree-of-freedom benchtop fluidics-driven catheter system was built to validate the proposed ANN. The proposed ANN-based shape sensing approach was evaluated on a 40-mm-long steerable continuum robot in both 3-D free-space and 2-D constrained environments, yielding an average shape sensing error of 0.24 and 0.49 mm, respectively. With these results, the superiority of the proposed approach compared to the recent model-based shape sensing method was demonstrated.

Robotic comanipulation provides a cost-effective solution to telesurgery when remote operation is not strictly necessary. Within the field of laparoscopic surgery, the comanip-ulation scenario is only recently being exploited commercially in the form of lightweight backdrivable systems. A passive wrist backdrivable robot does not require preoperative alignment with the incision that acts as a fulcrum around which the laparoscopic instrument pivots. Moreover, backdrivable systems can be comanipulated by the user without the need for expensive force sensors. Unfortunately, most backdrivable systems only provide limited accuracy when measuring the end effector pose from their joint encoders. Accurate knowledge of the end effector pose is required to estimate the the instrument tip and fulcrum position. This work presents a robust method to improve localisation of the pose of the end effector of a backdrivable robot. The method fuses optical tracking with robot proprioception by means of an unscented Kalman filter and is robust against intermittent occlusions of the line of sight. The algorithm is experimentally validated by analyzing its initialization behavior and accuracy when estimating the instrument tip and fulcrum position. An accuracy of$1.58\pm 0.157$mm and$0.699\pm 0.389$mm is achieved when estimating the instrument tip and fulcrum position respectively, which makes the algorithm suitable for advanced guidance schemes in comanipulation robotic surgery.

Force sensing is highly desirable in minimally invasive medical applications, since this feature shows great potential for reducing tissue damage and enhancing manipulation safety. However, embedding force sensors in medical devices is challenging and costly. This article explores the possibility to use shape sensing as a measure to extract force information. In this work, a model-based approach that allows simultaneous shape and force sensing is proposed. Shape information is reconstructed employing a multicore fiber with fiber Bragg grating sensors spaced over the fiber length. This fiber is capable of distributed 3D shape sensing. It is shown how by making use of extended Kalman filter and a mechanics model of the flexible instrument, it becomes possible to estimate both the magnitudes and locations of externally applied forces. Experiments were carried out to validate the proposed method for both one and two external forces applied at arbitrary locations in different directions on a flexible instrument. Results show that one-directional force magnitude and location can be estimated with an average error of 23.08 mN (15.39%) and 11.06 mm (6.51%), respectively. For two-directional forces, results of the load near the base show an average error of 52.01 mN (30.59%) for the magnitude and 29.24 mm (17.20%) for the location. For the load applied simultaneously near the tip, the mean magnitude error is 16.79 mN (11.19%) and the average location error is 10.18 mm (5.99%). The force sensing algorithm can run in real time with an approximate frequency of 59 Hz. In these experiments, it can be observed that the force-sensing accuracy, which depends on the sensitivity of the shape of flexible instruments with respect to the external force, can vary drastically in function of the force application point and force direction.

At the moment, surgeons struggle curing a widespread eye disease known as retinal vein occlusion where clots obstruct the retinal vessels. Latter vascular disorder involves black spots in people's eyesight and lead eventually to blindness. A recent promising treatment consists in flushing a thrombolytic agent inside the clotted retinal vessels. The surgery implies puncturing vessels ranging from 50 to 400 microns diameter on the backside of the eye, namely the retina. Latest research succeeded in tackling several challenges around this operation: the surgeon's hand tremor and the high precision required amongst other requirements. Despite several breakthroughs, the surgeon only relies on a microscope to perform the surgery through the patient eye's lens, giving poor depth perception to properly puncture the retinal vessels. This way, the surgeon is most likely to pierce through the vessel and inject the thrombolytic drug under the retina, which would endanger the person's eyesight. In this paper, we investigate the use of a novel bio-impedance sensor developed for eye surgery. Together with this new sensor, a detection algorithm has been developed to detect the puncture and double puncture events to give a feedback to the operator of the system. As far as we are aware of, such technology doesn't exist yet in eye surgery to tackle the depth perception question. This paper aims at demonstrating the benefits of this technology.

Retinal Vein Occlusion is a common retinal vascular disorder which can cause severe loss of vision. Retinal vein cannulation followed by injection of an anti-coagulant into the affected vein is a promising treatment. However, given the scale and fragility of the surgical workfield, this procedure is considered too high-risk to perform manually. A first successful robot-assisted procedure has been demonstrated. Even though successful, the procedure remains extremely challenging. This paper aims at providing a solution for the limited perception of instrument-tissue interaction forces as well as depth estimation during retinal vein cannulation. The development of a novel combined force and distance sensing cannulation needle relying on Fiber Bragg grating (FBG) and Optical Coherence Tomography (OCT) A-scan technology is reported. The design, the manufacturing process, the calibration method, and the experimental characterization of the produced sensor are discussed. The functionality of the combined sensing modalities and the real-time distance estimation algorithm are validated respectively on in-vitro and ex-vivo models.

Symbolic reasoners generate plans which are often not exploiting the robot capabilities and are sensitive to runtime disturbances. This work proposes a scheduler as an interface between a discrete, symbolic plan and a motion control based on constraint optimization. Acting as a local reasoner, the scheduler valuates a set of predicates to decide when an action will be executed. Given a task specification which describes how the action should be realized, the scheduler configures the controller at runtime. A demonstration will be provided considering an “open drawer” scenario.

This paper introduces a framework for constraint-based force/position control of robots that exhibit large nonlinear structural compliance and that undergo large deformations. Controller synthesis follows hereto the principles of the Task Frame and instantaneous Task Specification using Constraints (iTaSC) formalisms. iTaSC is found particularly suitable due to its ability to express and combine control tasks in a natural way. Control tasks can be formulated as combinations of target positions, velocities, or forces expressed in an arbitrary number and type of coordinate frames. The proposed framework is applied to a mixed mechatronic system composed of a traditional rigid-link robot whose end-effector is a continuum (flexible) link. A selection of different position/force control tasks is prepared to demonstrate the validity and general nature of the proposed framework.

This work aims to extend the application field of the constraint-based control framework called iTaSC (instantaneous task specification using constraints) toward manipulation tasks. iTaSC offers two advantages with respect to other methods: the ability to specify tasks in different spaces (and not only in Cartesian coordinates as for the Task Frame Formalism), and the treatment of geometric uncertainties. These properties may be very useful within a manipulation context, where tasks are executed by robots with many degrees of freedom, which calls for some degree of abstraction; by choosing a suitable set of coordinates, it is possible to reduce the complexity and the number of constraints that fully describe such tasks; in addition, controlling only the subspace that is needed to fulfil a task allows us to use the remaining degrees of freedom of the robot system to achieve secondary objectives. This paper discusses the instruments and techniques that can be employed in manipulation scenarios; in particular it focuses on aspects like the specification of a grasp and control of the stance of the robotic arm. iTaSC offers the possibility of specifying a grasp. While this approach allows for very fine control of a grasping task, in most cases a less fine-grain specification suffices to guarantee a successful execution of the grasping action. To this end synergy-based grasp specification is formulated within iTaSC. We also show how to take into account secondary objectives for the arm stance. In particular we consider, as an example, the manipulability index along a given direction. Such indexes are maximised by exploring the null space of the other tasks. The proposed approach is demonstrated by means of simulations, where a robotic hand grasps a cylindrical object.

This work aims to extend the application field of the constraint-based control framework called iTaSC (instantaneous task specification using constraints) toward tasks where physical interaction between the robot and the environment, or a human, is contemplated. iTaSC, in its original formulation, allows for a systematic derivation of control schemes from task descriptions; tasks are defined as constraints enforced on outputs (e.g. distances, angles), and the iTaSC control takes care to fulfil such constraints by computing desired velocities to be commanded to the robot(s) joints. This approach, being based on a velocity resolution scheme, principally addresses tasks where positioning is the main issue. However, tasks that involve contacts with the environment or with the user, either desired or accidental, can be considered as well, taking advantage of impedance control, when position is controlled, or with force control. This paper describes the implementation of force tasks, and, by the combination of conflicting force and position tasks, impedance control, within the iTaSC formalism. This result is achieved by taking advantage of an approximate physical modelling of the robotic system and the environment. The proposed control scheme is tested by means of experiments where constraints on forces and/or positions described in cylindrical coordinates are imposed on a Kuka LWR arm.

Combining tasks, melding their activities in sequence and in parallel, in order to achieve the desired goal is a challenging research topic. In many practical applications, tasks could be fulfilled even though the robot does not achieve a perfect matching with a given quantitative objective. Thus, it could be possible to carry out other activities, mildly conflicting with the current goal, yet providing benefits for the overall execution of a complex sequence of tasks. This paper proposes i) a criteria to evaluate the execution of a monitored task, ii) an enriched task specification to express the tolerance with which a goal is fulfilled, so that iii) a sequence of tasks can be executed concurrently, on the basis of monitored quantities. Furthermore, the paper reports a classification of conflicting scenarios for a finer selection of the scheduler policy. Finally, results from experimental scenarios show the potential benefits of the proposed methodology.

This work aims to extend the constraint-based formalism iTaSC for scenarios where physical human-robot interaction plays a central role, which is the case for e.g. surgical robotics, rehabilitation robotics and household robotics. To really exploit the potential of robots in these scenarios, it should be possible to enforce force and geometrical constraints in an easy and flexible way. iTaSC allows to express such constraints in different frames expressed in arbitrary spaces and to obtain control setpoints in a systematic way. In previous implementations of iTaSC, industrial velocity-controlled robots were considered. This work presents an extension of the iTaSC-framework that allows to take advantage of the back-drivability of a robot thus avoiding the use of force sensors. Then, as a casestudy, the iTaSC-framework is used to formulate a (positionposition) teleoperation scheme. The theoretical findings are experimentally validated using a PR2 robot.

In this paper, we deal with several aspects related to the control of tendon-based actuation systems for robotic devices. In particular, the problems that are considered in this paper are related to the modeling, identification, and control of tendons sliding on curved pathways, subject to friction and viscoelastic effects. Tendons made in polymeric materials are considered, and therefore, hysteresis in the transmission system characteristic must be taken into account as an additional nonlinear effect because of the plasticity and creep phenomena typical of these materials. With the aim of reproducing these behaviors, a viscoelastic model is used to model the tendon compliance. Particular attention has been given to the friction effects arising from the interaction between the tendon pathway and the tendon itself. This phenomenon has been characterized by means of a LuGre-like dynamic friction model to consider the effects that cannot be reproduced by employing a static friction model. A specific setup able to measure the tendon's tension in different points along its path has been designed in order to verify the tension distribution and identify the proper parameters. Finally, a simple control strategy for the compensation of these nonlinear effects and the control of the force that is applied by the tendon to the load is proposed and experimentally verified.

This paper presents the latest results in the development of the low-level controller of the robotic hand UBH-IV (University of Bologna Hand, version IV). In particular, the friction effects acting at joint level have been rendered by means of a LuGre-like model and a procedure for the identification of the friction model parameters is described. With the aim of providing an online estimation of the effects due to the interaction of the robotic hand with the environment, a controller able to evaluate the overall external torque acting on the finger joints and to discern between friction and torques generated by the external interaction force without using direct measures of the contact forces is proposed. The identification and control tests are carried over on an experimental setup composed by a single finger phalanx, manufactured with the same material and techniques of the hand itself.

This paper reports the modeling activities related to the development of an innovative tendon-driven robotic finger, designed as the fundamental element of a new biologically-inspired artificial hand. The finger is realized in plastic material by means of 3D-printing, a production process that allows a remarkable simplification of the mechanical design. Through 3D-printing, we were able to easily implement solutions that could be very difficult, if not impossible, to obtain with conventional manufacturing. A detailed simulation model of the robotic finger has been developed with the aim not only of designing and testing suitable control strategies for the finger, but also of investigating the benefits and the flaws of particular design solutions. As a matter of fact, this approach to design and realization of robotic fingers, that fulfills the requirements in terms of compactness, integration and simplified assembly, has a significant drawback in frictional phenomena on both tendons and joints. For this reason, an adapted LuGre friction model is proposed in order to simulate and study the finger behavior.

In this paper, the characterization of the force distribution along a tendon sliding on a curved pathway, subject to friction and visco-elastic effects, is investigated. In order to have a better understanding of the system behavior, a specific setup able to measure tension forces in different points along the tendon's path has been built. Experimental data collected by measuring the tendon tension forces during both the pulling and the release phase are presented, and theoretical models reproducing the tendon behavior with increasing fidelity are proposed. In particular, the friction arising from the interaction between the tendon pathway and the tendon itself is characterized by means of a LuGre-like dynamic friction model. The introduction of a dynamic friction model allows to reproduce in simulation some effects arising during experimental activities that cannot be reproduced employing an equivalent static friction model. Moreover, the adoption of tendons made by polymeric fibers introduces hysteresis in the tendon transmission characteristic due to the plasticity and creep phenomena typical of these materials. With the aim of reproducing this behavior, a visco-elastic model is used for modeling the tendon compliance.

Tendon-based transmission systems present many positive aspects and greatly simplify the mechanical design of small robotic devices, such as robotic fingers. On the other hand, they introduce several nonlinear effects that must be properly considered by the control algorithms to achieve a suitable performance level in the regulation of the finger joint torques. In this paper, the model of the tendons-based driving system and of the nonlinear effects arising from the use of sliding paths instead of pulleys for the tendon routing are discussed, and control algorithms aiming at compensating these nonlinearities are presented. Both models and control algorithms have been validated by experiments. In particular, in order to gain a better insight on the force distribution along the tendon, an experimental setup for the measurement of the tension in some intermediate points has been developed. After the identification of the tendon characteristics, a suitable control law for the compensation of the nonlinear effects due to the friction acting on the transmission system has been applied. The proposed compensation scheme is based on a sliding-mode controller with boundary layer, where the boundary threshold is modulated as a function of the desired tendon tension.

The time domain passivity framework is attracting interest as a method for granting stability in both telerobotics and haptic contexts; this paper employs this approach in order to introduce a novel concept, the Bilateral Energy Transfer for haptic telepresence. Loosely speaking, the Bilateral Energy Transfer is the straightforward transfer of energy between the two opposite sides of a teleoperation network, the master and slave robots. In an ideal telepresence scenario master and slave robots behave as rigid connected masses [1], and their power exchange is lossless; conversely, realistic scenarios include sources of energy leaks, i.e. elements that modify the power flows in the network. Moreover, if energy leaks have an active nature, they become source of instability for the system. This work isolates two sources of instability normally present in a teleoperation system, i.e. the delayed communication channel and robot velocity estimation based on digital position acquisition. These energy leaks are counterbalanced by two independent controllers, whose design is based on energetic consideration, and whose employment allows to achieve the Bilateral Energy Transfer. The presented arguments are sustained by simulations and experiments.

In this paper, two problems related to the simulation of virtual environments for haptic systems are considered. The first problem is how to simulate, in discrete time and with low computational effort, dynamic systems in order to preserve their passivity properties. As a matter of fact, simulation of complex systems in real time may lead to undesired effects, like unstable behaviours of the haptic interface, if proper care is not given to the definition of the simulation algorithm. An algorithm is presented here able to maintain the passivity properties of the physical (simulated) system with a reduced computational complexity. The second problem discussed in this paper is the interconnection of algorithms running at different frequencies, i.e., the control algorithm of the haptic interface (running typically at high frequency) and the algorithm simulating the virtual environment (running at lower frequency). A proper software interface, able to connect these two algorithms in an energetic-consistent manner, is presented and discussed. The general framework of both these techniques is the passivity theory and the so-called port-Hamiltonian formalism.