In ophthalmic surgery, particularly the procedures involving the posterior segment, clinicians face significant challenges in maintaining precise control of handheld instruments to peel fragile membranes without damaging the surrounding healthy fundus tissue, even for seasoned clinicians employing specialised ophthalmic surgical robots. The implementation of autonomous control in robot-assisted surgical systems holds promise for overcoming these obstacles and simplifying intricate surgical tasks. This paper introduces an autonomous control framework, integrating a Tip Detection Network (TDN) with a Proximal Policy Optimization (PPO) network, designed to autonomously navigate the Tip of the Surgical Instrument (ToSI) towards the intended lesion site in a real-world scenario. Results indicate that the accuracy of the TDN module in detecting the ToSI position in images of varying sizes can be reliably maintained within a 4.6-pixel range. The autonomous control deviation for the PPO module ranges between [0.6585μm, 7.995μm], with an average discrepancy of 5.118μm. The communication frequency across the modules is maintained at 35.7 Hz. In physical environments, the TDN-PPO framework adeptly navigates the ToSI to autonomously and precisely converge on the preset Target Lesion (TL) macular hole, maintaining a tip-to-target distance error within a margin of 2.094 pixels (38μm). Throughout the autonomous navigation phase, the maximal contact force between the ToSI and the TL is capped at 41.1 mN, aligning with the upper threshold for contact force between the ToSI and tissue prescribed in clinical surgical settings.

In ophthalmic surgery, especially in posterior segment procedures, clinicians face significant challenges, like the inherent tremor of the surgeon’s arm, restricted visibility, and heavy reliance on the surgeon’s skills for precise control of hand-held tools during micro-surgical movements. Automatic control of robotic-assisted ophthalmic surgical systems has the potential to overcome these challenges, simplifying complex surgical procedures. This paper proposes a novel image-guided automatic control method for an Ophthalmic micro-Surgical Robot (OmSR), specifically designed for posterior segment eye surgery. The method relies on forceps shadow tracking. The paper introduces a tip detection network (Net-SR), which accurately calculates the coordinates of the Tips of Surgical Forceps (ToSF) and Tips of Shadow (ToS) to enable automatic navigation. Additionally, through the Non-Uniform Rational B-Spline (NURBS) curve interpolation and speed look-ahead algorithm, dense and time-continuous data points are obtained to improve control accuracy and smoothness. The accuracy of the Net-SR network and motion of the ToSF, and the effectiveness of the proposed automatic controller are experimentally evaluated. Results demonstrate a significant 98.21% improvement in the Net-SR network accuracy over the normal keypoint detection network. The use of the speed look-ahead algorithm leads to a notable 41.7% improvement in optimal speed, and the ToSF successfully reaches the target lesion with vision-based navigation and no overscale motion. Note to Practitioners—The practical problem that motivated this research is the need for safer and more efficient surgical procedures, focusing on minimizing the risk of fundus tissue damage associated with intraoperative surgical instruments. To overcome challenges related to handheld and tele-operated control, we explore automatic control as a promising solution. In this paper, the tip of the instrument can consistently and accurately reach the target lesion with high precision and no overscale motion, allowing for deskilling of complex and repetitive tasks. This capability holds potential for the clinical needle insertion operation and membrane peeling operation. The proposed control methods can also be extended to other surgical procedures.

Screening of the lower gastrointestinal (GI) tract is of paramount importance for the early detection of precancerous lesions in the intestine, with an impact on reducing the high death rate of patients affected by cancer worldwide. Colonoscopy, i.e. standard procedure for screening the colon, is effective in reducing the incidence of colorectal cancer worldwide, nonetheless, this procedure remains an invasive method of screening, that typically causes discomfort and requires sedation for the patient. The SoftSCREEN system, a tethered robotic capsule designed for colonoscopy, aims to enable minimally invasive diagnosis of intestinal diseases through its innovative design that incorporates elastic tracks for locomotion and inflatable toroidal chambers for adaptable geometry to match the local lumen of the GI tract. After demonstrating the viability of the proposed design in a large-scale proof of concept in our previous work, the authors present here a miniaturised version of the SoftSCREEN system. We assess its performance in multiple phantom tests and evaluate the effect of pressure regulation on its locomotion. The conducted extensive tests demonstrate the capability of the soft robot to move inside intricate passages, capture internal images, and adjust its geometry to optimise traction. The results underscore the potential of the proposed design, offering promising advancements in the development of a robotic platform for efficient front-wheel locomotion and accurate intestinal screening.

Soft robotics, an emerging domain in modern robotics, introduces innovative possibilities alongside challenges in controllability, particularly with multi-degree inflatable actuators. We present a novel manipulation method using underactuated soft fingers that addresses these challenges by harnessing symmetry breaking. Central to our approach is the mechanism of self-organization within a ring actuator equipped with five fingers. Typically considered a drawback, we exploit the actuator’s buckling behavior to facilitate in-hand manipulation. This strategic utilization enables object motion in both clockwise and counterclockwise directions via system perturbations and adjustments in frequency and duty cycle parameters. Employing the self-organizing properties of our actuator, our method is empirically validated through simulations and real actuator experiments, demonstrating the system’s ability in manipulating objects by leveraging the inherent flexibility and morphological advantages. The design enables two degrees of freedom with minimal input, allowing objects to rotate due to the actuator’s self-organizing actions. This simplification of control mechanisms is essential for soft robotics manipulation. Our findings indicate that control systems in soft robotics can be significantly simplified, harnessing the adaptable behavior inherent in its morphology.

Tumors affecting abdominal organs rank among the deadliest malignancies. In this context, Magnetic Resonance Imaging (MRI) serves as an effective diagnostic tool with a strong potential to support image-guided minimally invasive interventions for treating these tumours, offering an ionizing-radiation-free medical modality. MRI provides exceptional soft tissue contrast and multi-angle imaging, enabling accurate intraoperative localisation of target tumours within these vital organs. Nevertheless, MRI-guided minimally invasive interventions still encounter significant challenges due to the strong magnetic field environment and the narrow and deep bore of MRI machines. This paper proposes a novel MR safe 5-degrees-of-freedom (DoFs) parallel table-mounted double-arch needle insertion robot with a scissor-folding mechanism (SFM) for abdominal interventions. The proposed robot is designed to fit a standard 70-cm MRI bore. Initial evaluation experiments indicate mean errors of 3.14 mm for the proposed robotic arch and 2.23 mm for the full needle insertion robot, respectively. Additionally, preliminary testing of the system in an MRI environment resulted in unaltered MRI imaging output, with negligible artefacts associated with the presence of the robot within the bore.

Soft robotic structures may play a major role in the 4th industrial revolution. Researchers have successfully demonstrated the advantages of soft robotics over traditional robots made of rigid links and joints in many application areas. Variable stiffness links (VSL) and joints (VSJ) have been investigated to achieve on-demand forces and, at the same time, be inherently safe in interactions with humans. However, a thorough characterisation of soft and rigid robotic components is still required. This paper investigates the influence of antagonistically actuated, stiffness-controllable joint-link units (JLUs) on the performance of collaborative robots (i.e. stiffness, load capacity, repetitive precision) and characterizes the difference compared with rigid units. A JLU is made of a combination of a VSL, a VSJ, and their rigid counterparts. Experimental results show that the VSL has minor differences in terms of stiffness (0.62 ∼ 0.95), output force (0.93 ∼ 0.94), and repetitive precision compared with the rigid link. For the VSJ, our results show a significant gap compared with the servo motor with regards to maximum stiffness (0.14 ∼ 0.21) and repetitive position precision (0.07 ∼ 0.25). However, similar performance on repetitive force precision and better performance on the maximum output force (1.54 ∼ 1.55 times) are demonstrated.

Minimally invasive surgery requires real-time tool tracking to guide the surgeon where depth perception and visual occlusion present navigational challenges. Although vision-based and external sensor-based tracking methods exist, fibre-optic sensing can overcome their limitations as they can be integrated directly into the device, are biocompatible, small, robust and geometrically versatile. In this paper, we integrate a fibre Bragg grating-based shape sensor into a soft robotic device. The soft robot is the pneumatically attachable flexible (PAF) rail designed to act as a soft interface between manipulation tools and intra-operative imaging devices. We demonstrate that the shape sensing fibre can detect the location of the tools paired with the PAF rail, by exploiting the change in curvature sensed by the fibre when a strain is applied to it. We then validate this with a series of grasping tasks and continuous US swipes, using the system to detect in real-time the location of the tools interacting with the PAF rail. The overall location-sensing accuracy of the system is 64.6%, with a margin of error between predicted location and actual location of 3.75 mm.

During laparoscopic surgery, tissues frequently need to be retracted and mobilized for manipulation or visualisation. State-of-the-art robotic platforms for minimally invasive surgery (MIS) typically rely on rigid tools to interact with soft tissues. Such tools offer a very narrow contact surface thus applying relatively large forces that can lead to tissue damage, posing a risk for the success of the procedure and ultimately for the patient. In this paper, we show how the use of Pneumatically Attachable Flexible (PAF) rail, a vacuum-based soft attachment for laparoscopic applications, can reduce such risk by offering a larger contact surface between the tool and the tissue. Ex vivo experiments are presented investigating the short- and long-term effects of different levels of vacuum pressure on the tissues surface. These experiments aim at evaluating the best trade-off between applied pressure, potential damage, task duration and connection stability. A hybrid control system has been developed to perform and investigate the organ repositioning task using the proposed system. The task is only partially automated allowing the surgeon to be part of the control loop. A gradient-based planning algorithm is integrated with learning from teleoperation algorithm which allows the robot to improve the learned trajectory. The use of Similar Smooth Path Repositioning (SSPR) algorithm is proposed to improve a demonstrated trajectory based on a known cost function. The results obtained show that a smoother trajectory allows to decrease the minimum level of pressure needed to guarantee active suction during PAF positioning and placement.

Soft robotic structures might play a major role in the 4thindustrial revolution. Researchers have successfully demonstrated advantages of soft robotics over traditional robots made of rigid links and joints in several application areas including manufacturing, healthcare and surgical interventions. However, soft robots have limited ability to exert higher forces when it comes to interaction with the environment, hence, change their stiffness on demand over a wide range. One stiffness mechanism embodies tendon-driven and pneumatic air actuation in an antagonistic way achieving variable stiffness values. In this paper, we apply a beam-mechanics-based model to this type of soft stiffness controllable robot. This mathematical model takes into account the various stiffness levels of the soft robotic manipulator as well as interaction forces with the environment at the tip of the manipulator. The analytical model is implemented into a robotic actuation system made of motorised linear rails with load cells (obtaining applied forces to the tendons) and a pressure regulator. Here, we present and analyse the performance and limitations of our model.

Nowadays, the field of industrial robotics focuses particularly on collaborative robots that are able to work closely together with a human worker in an inherently safe way. To detect and prevent harmful collisions, a number of solutions both from the actuation and sensing sides have been suggested. However, due to the rigid body structures of the majority of systems, the risk of harmful collisions with human operators in a collaborative environment remains. In this paper, we propose a novel concept for a collaborative robot made of Variable Stiffness Links (VSLs). The idea is to use a combination of silicone based structures and fabric materials to create stiffness-controllable links that are pneumatically actuated. According to the application, it is possible to change the stiffness of the links by varying the value of pressure inside their structure. Moreover, the pressure readings from the pressure sensors inside the regulators can be utilised to detect collisions between the manipulator body and a human worker, for instance. A set of experiments are performed with the aim to assess the performance of the VSL when embedded in a robotic manipulator. The effects of different loads and pressures on the workspace of the manipulator are evaluated together with the efficiency of the collision detection control system and hardware.

Proximity and distance estimation sensors are broadly used in robotic hands to enhance the quality of grasping during grasp planning, grasp correction and in-hand manipulation. This paper presents a fiber optical proximity sensor that is integrated with a tactile sensing fingertip of a robotic hand of a mobile robot. The distance estimation of proximity sensors are typically influenced by the reflective properties of an object, such as color or surface roughness. With the approach proposed in this paper, the accuracy of the proximity sensor is enhanced using the information collected by the vision system of the robot. A camera is employed to obtain RGB values of the object to be grasped. Further on, the data obtained from the camera is used to obtain the correct calibration for the proximity sensor. Based on the experimental evidence, it is shown that our approach can be effectively used to reduce the distance estimation error.

This paper presents a methodology for the development of a multi-axis force/torque sensor based on optoelectronic technology. The advantages of using this sensing principle are the low manufacturing costs, the simple fabrication, and the immunity to electrical noise. The force/ torque sensor makes use of six optical sensors: each sensor measures the displacement of a reflective surface that moves integrally with a simply-supported beam. The proposed mechanical structure allows for a variety of shapes on the mechanical structure to be easily adaptable to many robot applications. In this paper, we present a five-axis force/torque sensor based on this optoelectronic principle. To measure force/torque components, two identical three-DoF force/torque sensor structures (comprised of three beams) are mounted on top of each other. Photo sensors and mirrors are fixed inside the structure to measure the six beam deflections. In this paper, we describe the sensor structure, design, fabrication, calibration, and verify our sensor development methodology.

This paper proposes a soft, inflatable manipulator that is antagonistically actuated by tendons and pneumatics. The combination of the two actuation mechanisms in this antagonistic robot structure is inspired by the octopus which uses its longitudinal and transversal muscles to steer, elongate, shrink and also stiffen its continuum arms. By “activating” its antagonistic muscle groups at the same time, the octopus can achieve multiple motion patterns as well as stiffen their arms. Being organized in a similar fashion, our robot manipulator uses, on the one hand, pneumatic actuation and, on the other hand, tendon-based actuation - one opposing the other, achieving an overall antagonistic actuation framework. Controlling the pressure inside the robot while at the same time controlling the tendons' displacements, the robot can be moved into a wide range of configurations while simultaneously controlling the arm's stiffness. This paper builds on earlier work by the authors: Here, we present a new conic-shaped manipulator structure and the control architecture. Using a constant curvature model, we have derived an approach suitable for controlling the robot manipulator. The manipulator's reachable workspace is analyzed and proof-of-concept experiments were conducted to show the robot's stiffness control and motion abilities.

This paper presents an innovative hand-held device able to compute stiffness when interacting with a soft object. The device is composed of four linear indenters and a USB camera. The stiffness is computed in real-time, tracking the movements of spherical features in the image of the camera. Those movements relate to the movements of the four indenters when interacting with a soft surface. Since the indenters are connected to springs with different spring constants, the displacement of the indenters varies when interacting with a soft object. The proposed multi-indenting device allows measuring the object's stiffness as well as the pan and tilt angles between the sensor and the surface of the soft object. Tests were performed to evaluate the accuracy of the proposed palpation mechanism against commercial springs of known stiffness. Results show that the accuracy and sensitivity of the proposed device increases with the softness of the examined object. Preliminary tests with silicon show the ability of the sensing mechanism to characterize phantom soft tissue for small indentation. It is noted that the results are not affected by the orientation of the device when probing the surface. The proposed sensing device can be used in different applications, such as external palpation for diagnosis or, if miniaturized, embedded on an endoscopic camera and used in Minimally Invasive Surgery (MIS).

It is well known that tactile information can be used to enhance the quality of grasping. Therefore, new technological solutions for sensing in grasping are needed. This paper presents an optical based fingertip sensor that measures both interaction forces and proximity between fingertip and environment. The combination of multiple sensing modalities in the tip of a finger can significantly improve grasping and manipulation capabilities. In this work we present the design and the required calibration of individual sensing elements, and of the integrated fingertip sensor developed for a 3-fingered metamorphic robotic hand. Emulated grasping experiments, using a pinch grip, were performed to illustrate the concept and validate the performance of the developed sensing system. As a result, it was possible to determine the sensor position with respect to an object during approach, contact and grasp.

This paper explores a new hybrid actuation principle combining pneumatic and tendon-driven actuators for a soft robotic manipulator. The fusion of these two actuation principles leads to an overall antagonistic actuation mechanism whereby pneumatic actuation opposes tendon actuation - a mechanism commonly found in animals where muscles can oppose each other to vary joint stiffness. We are taking especially inspiration from the octopus who belongs to the class of Cephalopoda; the octopus uses its longitudinal and transversal muscles in its arms to achieve varied motion patterns; activating both sets of muscles, the octopus can control the arm stiffness over a wide range. Our approach mimics this behavior and achieves comparable motion patterns, including bending, elongation and stiffening. The proposed method combines the advantages of tendon-driven and pneumatic actuated systems and goes beyond what current soft, flexible robots can achieve: because the new robot structure is effectively an inflatable, sleeve, it can be pumped up to its fully inflated volume and, also, completely deflated and shrunk. Since, in the deflated state, it comprises just its outer “skin” and tendons, the robot can be compressed to a very small size, many times smaller when compared to its fully-inflated state. In this paper, we describe the mechanical structure of the soft manipulator. Proof-of-concept experiments focus on the robot's ability to bend, to morph from completely shrunk to entirely inflated as well as to vary its stiffness.